AutoGuess tests (#1650)

* whitespace

* AutoGuess remove dot suffix in names

* .gitignore update

* test: added autoguess test suite

* github workflow to run autoguess test when appropriate

* git clone unavailable tokenizer configs rather than committing to repo

* fix link to included tokenizer configs

* skip storing downloaded tokenizer configs

* typo

* minor fixes

* clean-up

* limit workflow to trigger from experimental branch

---------

Co-authored-by: Concedo <39025047+LostRuins@users.noreply.github.com>

.github/workflows/test-autoguess.yaml

@@ -0,0 +1,30 @@
+name: AutoGuess Tests
+
+on:
+  pull_request:
+    branches:
+      - concedo_experimental
+    paths:
+      - 'kcpp_adapters/AutoGuess.json'
+
+jobs:
+  test-autoguess:
+    runs-on: ubuntu-latest
+
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v4
+
+    - name: Set up Python
+      uses: actions/setup-python@v4
+      with:
+        python-version: '3.x'  # Adjust to your preferred Python version
+
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install requests
+        git clone https://github.com/kallewoof/gated-tokenizers.git tests/gated-tokenizers
+
+    - name: Run AutoGuess tests
+      run: python tests/test_autoguess.py