Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,184 @@
+---
+library_name: coreml
+license: cc-by-4.0
+tags:
+  - coreml
+  - asr
+  - speech-recognition
+  - streaming
+  - nemotron
+  - nvidia
+  - rnnt
+  - fastconformer
+  - apple-silicon
+  - neural-engine
+language:
+  - en
+base_model: nvidia/nemotron-speech-streaming-en-0.6b
+pipeline_tag: automatic-speech-recognition
+---
+
+# Nemotron Speech 600M — CoreML (Streaming)
+
+Native CoreML conversion of [nvidia/nemotron-speech-streaming-en-0.6b](https://huggingface.co/nvidia/nemotron-speech-streaming-en-0.6b), a 600M-parameter streaming ASR model using FastConformer encoder + RNNT decoder. Optimized for Apple Neural Engine (ANE) on Apple Silicon.
+
+Converted directly from the original NeMo checkpoint via coremltools, with 3-level numerical validation against the PyTorch reference.
+
+Sibling project: **[danielbodart/nemotron-speech-600m-onnx](https://huggingface.co/danielbodart/nemotron-speech-600m-onnx)** — ONNX Runtime version for Linux (CUDA + CPU).
+
+## Performance
+
+| Metric | Value |
+|--------|-------|
+| Encoder ANE utilization | **93%** (1486/1597 ops on Neural Engine) |
+| Inference speed | **~15.5x realtime** on Apple Silicon |
+| Encoder CPU ops | 7% (softmax, attention masking — see config.json for breakdown) |
+| Decoder | 100% CPU (CPU_ONLY compute units) |
+
+No manual ANE optimizations applied — coremltools compiler routes ops automatically.
+
+## Available Precisions
+
+| Variant | Encoder | Decoder | Total | Compute Units | Notes |
+|---------|---------|---------|-------|---------------|-------|
+| `fp16/` | 1.1 GB | 17 MB | ~1.1 GB | Encoder: CPU+ANE, Decoder: CPU | Recommended |
+
+Future: INT8 quantization via coremltools (can halve encoder size on ANE).
+
+## Model Architecture
+
+Two CoreML models (decoder and joint network are fused into one):
+
+| Model | Input | Output | Compute |
+|-------|-------|--------|---------|
+| **Encoder** | mel `[1, 128, 65]` + caches (FP32 in, FP16 out) | encoded `[1, 1024, 7]` + caches | CPU_AND_NE |
+| **Fused Decoder+Joint** | enc_frame `[1, 1024, 1]` + token `[1, 1]` + LSTM h,c (FP32 in, FP16 out) | logits `[1, 1025]` + LSTM h,c | CPU_ONLY |
+
+Mel spectrogram preprocessing runs on the host (not in CoreML).
+
+### Important: ANE stride padding
+
+CoreML output MLMultiArrays may have non-contiguous strides due to ANE alignment padding. For example, the encoder output `[1, 1024, 7]` may have physical strides `[32768, 32, 1]` instead of C-contiguous `[7168, 7, 1]`. **Callers must use stride-aware copy, not flat memcpy.**
+
+## Runtime Configuration
+
+All parameters needed to run the model are documented in [`config.json`](config.json), including I/O specs, cache shapes, the streaming protocol, and ANE profiling results.
+
+### Audio Preprocessing
+
+| Parameter | Value |
+|-----------|-------|
+| Sample rate | 16000 Hz |
+| Sample format | S16_LE (16-bit signed little-endian) |
+| Pre-emphasis | 0.97 |
+| FFT size | 512 |
+| Hop length | 160 samples (10ms) |
+| Window length | 400 samples (25ms) |
+| Window type | Hann |
+| Mel bands | 128 |
+| Mel norm | Slaney |
+| Mel layout | **Band-major** `[n_mels, n_frames]` (not frame-major) |
+
+### Encoder Streaming
+
+| Parameter | Value |
+|-----------|-------|
+| Chunk size | 56 mel frames (560ms audio) |
+| Pre-encode cache | 9 mel frames (prepended from previous chunk) |
+| Total input frames | 65 per chunk (56 + 9) |
+| Layers | 24 |
+| Dimension | 1024 |
+| `cache_last_channel` shape | `[1, 24, 70, 1024]` FP32 in, FP16 out (init zeros) |
+| `cache_last_time` shape | `[1, 24, 1024, 8]` FP32 in, FP16 out (init zeros) |
+| `cache_last_channel_len` | `[1]` int32 (init zero) |
+
+Feed cache outputs back as next chunk's cache inputs. Convert FP16 outputs to FP32 before feeding back (model expects FP32 inputs).
+
+### RNNT Decoder
+
+| Parameter | Value |
+|-----------|-------|
+| Blank token ID | 1024 |
+| Vocab size | 1025 (1024 tokens + blank) |
+| Max symbols per frame | 10 |
+| Prediction layers | 2 (LSTM) |
+| Prediction hidden | 640 |
+| `input_states_1/2` shape | `[2, 1, 640]` FP32 in, FP16 out (init zeros) |
+
+For each encoder output frame: feed single frame `[1, 1024, 1]` to decoder, argmax logits over 1025 vocab, if not blank emit token and loop (up to 10), if blank move to next frame. Feed decoder states back for next symbol/frame.
+
+## Files
+
+```
+config.json                       # Machine-readable runtime parameters + ANE profile
+
+fp16/
+├── encoder.mlmodelc/             # Pre-compiled encoder (load directly with MLModel)
+│   ├── model.mil
+│   ├── coremldata.bin
+│   └── weights/weight.bin
+├── encoder.mlpackage/            # Source encoder (for runtime compilation fallback)
+│   └── Data/com.apple.CoreML/
+│       ├── model.mlmodel
+│       └── weights/weight.bin
+├── decoder.mlmodelc/             # Pre-compiled fused decoder+joint
+├── decoder.mlpackage/            # Source fused decoder+joint
+└── metadata.json                 # Cache shapes, vocab size, model parameters
+```
+
+## Usage
+
+Download:
+
+```bash
+# Download compiled models (recommended)
+hf download danielbodart/nemotron-speech-600m-coreml fp16/ config.json --local-dir ./model
+
+# Download only .mlmodelc (skip .mlpackage to save space)
+hf download danielbodart/nemotron-speech-600m-coreml fp16/encoder.mlmodelc/ fp16/decoder.mlmodelc/ fp16/metadata.json config.json --local-dir ./model
+```
+
+Load with CoreML (Swift):
+
+```swift
+import CoreML
+
+let config = MLModelConfiguration()
+config.computeUnits = .cpuAndNeuralEngine
+
+let encoder = try MLModel(contentsOf: URL(fileURLWithPath: "model/fp16/encoder.mlmodelc"),
+                          configuration: config)
+
+let decConfig = MLModelConfiguration()
+decConfig.computeUnits = .cpuOnly
+let decoder = try MLModel(contentsOf: URL(fileURLWithPath: "model/fp16/decoder.mlmodelc"),
+                          configuration: decConfig)
+```
+
+Load with CoreML (Python/coremltools):
+
+```python
+import coremltools as ct
+
+encoder = ct.models.MLModel("model/fp16/encoder.mlpackage")
+decoder = ct.models.MLModel("model/fp16/decoder.mlpackage")
+```
+
+## Conversion Reproducibility
+
+All conversion and validation scripts are in the companion GitHub repo: **[danielbodart/nemotron-speech-600m-coreml](https://github.com/danielbodart/nemotron-speech-600m-coreml)**
+
+- `convert.py` — NeMo → CoreML conversion (wrap, trace, convert, compile)
+- `validate.py` — 3-level validation (wrapper equiv, CoreML vs PyTorch, end-to-end transcript)
+- `wrappers.py` — PyTorch wrappers (EncoderWrapper, FusedDecoderJointWrapper)
+
+Requires macOS with Apple Silicon, Python 3.10, coremltools 9.0b1.
+
+## Related
+
+- **[danielbodart/nemotron-speech-600m-onnx](https://huggingface.co/danielbodart/nemotron-speech-600m-onnx)** — ONNX Runtime version for Linux (CUDA + CPU, FP16/INT8)
+- **[nvidia/nemotron-speech-streaming-en-0.6b](https://huggingface.co/nvidia/nemotron-speech-streaming-en-0.6b)** — Original NeMo model
+
+## License
+
+The original model is licensed under [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/) by NVIDIA.