google
diff --git a/‎README.md‎
Lines changed: 49 additions & 115 deletions b/‎README.md‎
Lines changed: 49 additions & 115 deletions
@@ -6,7 +6,7 @@ foundation models from Google.
 For additional information about Gemma, see
 [ai.google.dev/gemma](https://ai.google.dev/gemma). Model weights, including
 gemma.cpp specific artifacts, are
-[available on kaggle](https://www.kaggle.com/models/google/gemma).
+[available on kaggle](https://www.kaggle.com/models/google/gemma-2).
 
 ## Who is this project for?
 
@@ -18,8 +18,8 @@ deployment-oriented C++ inference runtimes, which are not designed for
 experimentation, and Python-centric ML research frameworks, which abstract away
 low-level computation through compilation.
 
-gemma.cpp provides a minimalist implementation of Gemma-1, Gemma-2, Gemma-3, and
-PaliGemma models, focusing on simplicity and directness rather than full
+gemma.cpp provides a minimalist implementation of Gemma-2, Gemma-3, and
+PaliGemma-2 models, focusing on simplicity and directness rather than full
 generality. This is inspired by vertically-integrated model implementations such
 as [ggml](https://github.com/ggerganov/ggml),
 [llama.c](https://github.com/karpathy/llama2.c), and
@@ -53,7 +53,7 @@ Guidelines](https://opensource.google.com/conduct/).
 
 -   LLM
 
-    -   CPU-only inference for: Gemma 1-3, Griffin(SSM), PaliGemma 1-2.
+    -   CPU-only inference for: Gemma 2-3, Griffin(SSM), PaliGemma 2.
     -   Sampling with TopK and temperature.
     -   Backward pass (VJP) and Adam optimizer for Gemma research.
 
@@ -106,57 +106,20 @@ winget install --id Microsoft.VisualStudio.2022.BuildTools --force --override "-
 
 Visit the
 [Kaggle page for Gemma-2](https://www.kaggle.com/models/google/gemma-2/gemmaCpp)
-[or Gemma-1](https://www.kaggle.com/models/google/gemma/frameworks/gemmaCpp),
 and select `Model Variations |> Gemma C++`.
 
 On this tab, the `Variation` dropdown includes the options below. Note bfloat16
 weights are higher fidelity, while 8-bit switched floating point weights enable
 faster inference. In general, we recommend starting with the `-sfp` checkpoints.
 
-If you are unsure which model to start with, we recommend starting with the
-smallest Gemma-2 model, i.e. `2.0-2b-it-sfp`.
-
-Alternatively, visit the
-[gemma.cpp](https://huggingface.co/models?other=gemma.cpp) models on the Hugging
-Face Hub. First go the model repository of the model of interest (see
-recommendations below). Then, click the `Files and versions` tab and download
-the model and tokenizer files. For programmatic downloading, if you have
-`huggingface_hub` installed, you can also download by running:
-
-```
-huggingface-cli login # Just the first time
-huggingface-cli download google/gemma-2b-sfp-cpp --local-dir build/
-```
-
-Gemma-1 2B instruction-tuned (`it`) and pre-trained (`pt`) models:
-
-| Model name  | Description |
-| ----------- | ----------- |
-| `2b-it`     | 2 billion parameter instruction-tuned model, bfloat16 |
-| `2b-it-sfp` | 2 billion parameter instruction-tuned model, 8-bit switched floating point |
-| `2b-pt`     | 2 billion parameter pre-trained model, bfloat16 |
-| `2b-pt-sfp` | 2 billion parameter pre-trained model, 8-bit switched floating point |
-
-Gemma-1 7B instruction-tuned (`it`) and pre-trained (`pt`) models:
-
-| Model name  | Description |
-| ----------- | ----------- |
-| `7b-it`     | 7 billion parameter instruction-tuned model, bfloat16 |
-| `7b-it-sfp` | 7 billion parameter instruction-tuned model, 8-bit switched floating point |
-| `7b-pt`     | 7 billion parameter pre-trained model, bfloat16 |
-| `7b-pt-sfp` | 7 billion parameter pre-trained model, 8-bit switched floating point |
-
-> [!NOTE]
-> **Important**: We strongly recommend starting off with the `2b-it-sfp` model to
-> get up and running.
+> [!NOTE] **Important**: We strongly recommend starting off with the
+> `gemma2-2b-it-sfp` model to get up and running.
 
 Gemma 2 models are named `gemma2-2b-it` for 2B and `9b-it` or `27b-it`. See the
-`kModelFlags` definition in `common.cc`.
+`ModelPrefix` function in `configs.cc`.
 
 ### Step 2: Extract Files
 
-If you downloaded the models from Hugging Face, skip to step 3.
-
 After filling out the consent form, the download should proceed to retrieve a
 tar archive file `archive.tar.gz`. Extract files from `archive.tar.gz` (this can
 take a few minutes):
@@ -194,10 +157,9 @@ cmake --build --preset make -j [number of parallel threads to use]
 ```
 
 Replace `[number of parallel threads to use]` with a number - the number of
-cores available on your system is a reasonable heuristic.  For example,
-`make -j4 gemma` will build using 4 threads. If the `nproc` command is
-available, you can use `make -j$(nproc) gemma` as a reasonable default
-for the number of threads.
+cores available on your system is a reasonable heuristic. For example, `make -j4
+gemma` will build using 4 threads. If the `nproc` command is available, you can
+use `make -j$(nproc) gemma` as a reasonable default for the number of threads.
 
 If you aren't sure of the right value for the `-j` flag, you can simply run
 `make gemma` instead and it should still build the `./gemma` executable.
@@ -206,7 +168,8 @@ If you aren't sure of the right value for the `-j` flag, you can simply run
 > On Windows Subsystem for Linux (WSL) users should set the number of
 > parallel threads to 1. Using a larger number may result in errors.
 
-If the build is successful, you should now have a `gemma` executable in the `build/` directory.
+If the build is successful, you should now have a `gemma` executable in the
+`build/` directory.
 
 #### Windows
 
@@ -218,15 +181,17 @@ cmake --preset windows
 cmake --build --preset windows -j [number of parallel threads to use]
 ```
 
-If the build is successful, you should now have a `gemma.exe` executable in the `build/` directory.
+If the build is successful, you should now have a `gemma.exe` executable in the
+`build/` directory.
 
 #### Bazel
 
 ```sh
 bazel build -c opt --cxxopt=-std=c++20 :gemma
 ```
 
-If the build is successful, you should now have a `gemma` executable in the `bazel-bin/` directory.
+If the build is successful, you should now have a `gemma` executable in the
+`bazel-bin/` directory.
 
 #### Make
 
@@ -240,33 +205,21 @@ You can now run `gemma` from inside the `build/` directory.
 
 `gemma` has the following required arguments:
 
-Argument        | Description                  | Example value
---------------- | ---------------------------- | -----------------------
-`--model`       | The model type.              | `2b-it` ... (see below)
-`--weights`     | The compressed weights file. | `2b-it-sfp.sbs`
-`--weight_type` | The compressed weight type.  | `sfp`
-`--tokenizer`   | The tokenizer file.          | `tokenizer.spm`
-
-`gemma` is invoked as:
-
-```sh
-./gemma \
---tokenizer [tokenizer file] \
---weights [compressed weights file] \
---weight_type [f32 or bf16 or sfp (default:sfp)] \
---model [2b-it or 2b-pt or 7b-it or 7b-pt or ...]
-```
+Argument      | Description                  | Example value
+------------- | ---------------------------- | ---------------
+`--weights`   | The compressed weights file. | `2b-it-sfp.sbs`
+`--tokenizer` | The tokenizer file.          | `tokenizer.spm`
 
 Example invocation for the following configuration:
 
-- Compressed weights file `2b-it-sfp.sbs` (2B instruction-tuned model, 8-bit
-  switched floating point).
-- Tokenizer file `tokenizer.spm`.
+-   weights file `gemma2-2b-it-sfp.sbs` (Gemma2 2B instruction-tuned model,
+    8-bit switched floating point).
+-   Tokenizer file `tokenizer.spm` (can omit for single-format weights files
+    created after 2025-05-06, or output by migrate_weights.cc).
 
 ```sh
 ./gemma \
---tokenizer tokenizer.spm \
---weights 2b-it-sfp.sbs --model 2b-it
+--tokenizer tokenizer.spm --weights gemma2-2b-it-sfp.sbs
 ```
 
 ### RecurrentGemma
@@ -288,11 +241,9 @@ Step 1, and run the binary as follows:
 
 ### PaliGemma Vision-Language Model
 
-This repository includes a version of the PaliGemma VLM
-([paper](https://arxiv.org/abs/2407.07726),
-[code](https://github.com/google-research/big_vision/tree/main/big_vision/configs/proj/paligemma))
-and its successor PaliGemma 2 ([paper](https://arxiv.org/abs/2412.03555)). We
-provide a C++ implementation of the PaliGemma model family here.
+This repository includes a version of the PaliGemma 2 VLM
+([paper](https://arxiv.org/abs/2412.03555)). We provide a C++ implementation of
+the PaliGemma 2 model here.
 
 To use the version of PaliGemma included in this repository, build the gemma
 binary as noted above in Step 3. Download the compressed weights and tokenizer
@@ -303,8 +254,7 @@ and run the binary as follows:
 ```sh
 ./gemma \
 --tokenizer paligemma_tokenizer.model \
---model paligemma-224 \
---weights paligemma-3b-mix-224-sfp.sbs \
+--weights paligemma2-3b-mix-224-sfp.sbs \
 --image_file paligemma/testdata/image.ppm
 ```
 
@@ -346,26 +296,17 @@ from the multi-file format to the single-file format is available.
 ```sh
 io/migrate_weights \
   --tokenizer .../tokenizer.spm --weights .../gemma2-2b-it-sfp.sbs \
-  --model gemma2-2b-it --output_weights .../gemma2-2b-it-sfp-single.sbs
+  --output_weights .../gemma2-2b-it-sfp-single.sbs
 ```
 
-After migration, you can use the new weights file with gemma.cpp like this:
+After migration, you can omit the tokenizer argument like this:
 
 ```sh
 ./gemma --weights .../gemma2-2b-it-sfp-single.sbs
 ```
 
 ### Troubleshooting and FAQs
 
-**Running `./gemma` fails with "Failed to read cache gating_ein_0 (error 294) ..."**
-
-The most common problem is that the `--weight_type` argument does not match that
-of the model file. Revisit step #3 and check which weights you downloaded.
-
-Note that we have already moved weight type from a compile-time decision to a
-runtime argument. In a subsequent step, we plan to bake this information into
-the weights.
-
 **Problems building in Windows / Visual Studio**
 
 Currently if you're using Windows, we recommend building in WSL (Windows
@@ -376,8 +317,8 @@ configurations, see issues for active discussion.
 
 A common issue is that you are using a pre-trained model, which is not
 instruction-tuned and thus does not respond to instructions. Make sure you are
-using an instruction-tuned model (`2b-it-sfp`, `2b-it`, `7b-it-sfp`, `7b-it`)
-and not a pre-trained model (any model with a `-pt` suffix).
+using an instruction-tuned model (`gemma2-2b-it-sfp`) and not a pre-trained
+model (any model with a `-pt` suffix).
 
 **What sequence lengths are supported?**
 
@@ -387,11 +328,10 @@ sequences will be slow due to the quadratic cost of attention.
 
 **How do I convert my fine-tune to a `.sbs` compressed model file?**
 
-For PaliGemma (1 and 2) checkpoints, you can use
-python/convert_from_safetensors.py to convert from safetensors format (tested
-with building via bazel). For an adapter model, you will likely need to call
-merge_and_unload() to convert the adapter model to a single-file format before
-converting it.
+For PaliGemma 2 checkpoints, you can use python/convert_from_safetensors.py to
+convert from safetensors format (tested with building via bazel). For an adapter
+model, you will likely need to call merge_and_unload() to convert the adapter
+model to a single-file format before converting it.
 
 Here is how to use it using a bazel build of the compression library assuming
 locally installed (venv) torch, numpy, safetensors, absl-py, etc.:
@@ -405,22 +345,18 @@ ln -s $BAZEL_OUTPUT_DIR [...]/site-packages/compression
 python3 python/convert_from_safetensors.py --load_path [...].safetensors.index.json
 ```
 
-See also compression/convert_weights.py for a slightly older option to convert a
-pytorch checkpoint. (The code may need updates to work with Gemma-2 models.)
-
 **What are some easy ways to make the model run faster?**
 
 1.  Make sure you are using the 8-bit switched floating point `-sfp` models.
     These are half the size of bf16 and thus use less memory bandwidth and cache
     space.
-2.  If you're on a laptop, make sure power mode is set to maximize performance
+2.  Due to auto-tuning, the second and especially third query will be faster.
+3.  If you're on a laptop, make sure power mode is set to maximize performance
     and saving mode is **off**. For most laptops, the power saving modes get
     activated automatically if the computer is not plugged in.
-3.  Close other unused cpu-intensive applications.
-4.  On macs, anecdotally we observe a "warm-up" ramp-up in speed as performance
+4.  Close other unused cpu-intensive applications.
+5.  On macs, anecdotally we observe a "warm-up" ramp-up in speed as performance
     cores get engaged.
-5.  Experiment with the `--num_threads` argument value. Depending on the device,
-    larger numbers don't always mean better performance.
 
 We're also working on algorithmic and optimization approaches for faster
 inference, stay tuned.
@@ -452,10 +388,7 @@ $ ./gemma [...]
   __/ |                                    | |   | |
  |___/                                     |_|   |_|
 
-tokenizer                     : tokenizer.spm
-weights                       : 2b-it-sfp.sbs
-model                         : 2b-it
-max_generated_tokens          : 2048
+...
 
 *Usage*
   Enter an instruction and press enter (%C reset conversation, %Q quits).
@@ -493,7 +426,7 @@ For using the `gemma` executable as a command line tool, it may be useful to
 create an alias for gemma.cpp with arguments fully specified:
 
 ```sh
-alias gemma2b="~/gemma.cpp/build/gemma -- --tokenizer ~/gemma.cpp/build/tokenizer.spm --weights ~/gemma.cpp/build/gemma2-2b-it-sfp.sbs --model gemma2-2b-it --verbosity 0"
+alias gemma2b="~/gemma.cpp/build/gemma -- --tokenizer ~/gemma.cpp/build/tokenizer.spm --weights ~/gemma.cpp/build/gemma2-2b-it-sfp.sbs --verbosity 0"
 ```
 
 Replace the above paths with your own paths to the model and tokenizer paths
@@ -523,8 +456,8 @@ Let's break down the code:
 ### Incorporating gemma.cpp as a Library in your Project
 
 The easiest way to incorporate gemma.cpp in your own project is to pull in
-gemma.cpp and dependencies using `FetchContent`. You can add the following to your
-CMakeLists.txt:
+gemma.cpp and dependencies using `FetchContent`. You can add the following to
+your CMakeLists.txt:
 
 ```
 include(FetchContent)
@@ -593,9 +526,10 @@ submit a PR with a `README.md` edit.
 
 ## Acknowledgements and Contacts
 
-gemma.cpp was started in fall 2023 by [Austin Huang](mailto:austinvhuang@google.com)
-and [Jan Wassenberg](mailto:janwas@google.com), and subsequently released February 2024
-thanks to contributions from Phil Culliton, Paul Chang, and Dan Zheng.
+gemma.cpp was started in fall 2023 by
+[Austin Huang](mailto:austinvhuang@google.com) and
+[Jan Wassenberg](mailto:janwas@google.com), and subsequently released February
+2024 thanks to contributions from Phil Culliton, Paul Chang, and Dan Zheng.
 
 Griffin support was implemented in April 2024 thanks to contributions by Andrey
 Mikhaylov, Eugene Kliuchnikov, Jan Wassenberg, Jyrki Alakuijala, Lode