File size: 1,331 Bytes
0e3d284 6dac1ef ea8ec06 0e3d284 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
---
license: mit
---
Series of models to test the benefits of CoreML joint compression on iOS 18/macOS 15.
# mlp-*.mlpackage
Simple Up/Gate/Silu/Down MLP repeated four times with the Llama 2 7B dimensions.
All using 'CPU and Neural Engine' compute unit, measured in Xcode.
|Device|Model |Precision |Minimum (ms)|Median (ms)|
|:-- |:-- |:-- |--: |--: |
|M1 Max|mlp-float16 |float16 |19.30 |19.42 |
|M1 Max|mlp-4bit |4-bit LUT |5.93 |5.98 |
|M1 Max|mlp-2bit |2-bit LUT |5.92 |6.11 |
|M1 Max|mlp-4bit-int8|4-bit int8 LUT + A8|6.02 |6.31 |
|M1 Max|mlp-2bit-int8|2-bit int8 LUT + A8|6.00 |6.18 |
|M1 Max|mlp-int8-int8|W8A8 |9.78 |9.94 |
|M4 |mlp-4bit |4-bit LUT |- |4.19 |
|M4 |mlp-2bit |2-bit LUT |- |3.83 |
|M4 |mlp-4bit-int8|4-bit int8 LUT + A8|- |4.14 |
|M4 |mlp-2bit-int8|2-bit int8 LUT + A8|- |3.83 |
|M4 |mlp-int8-int8|W8A8 |- |8.18 |
# Download
```
huggingface-cli download \
--local-dir . \
--local-dir-use-symlinks False \
smpanaro/coreml-joint-compression-test \
--include "*.mlpackage/*"
``` |