Commit History

apply device-transfer patch from https://github.com/mosaicml/llm-foundry/pull/225/files
ec8bed8
unverified

Alex Birch commited on

prefer NamedTuple
ec8ea9d
unverified

Alex Birch commited on

gradient checkpointing for multi-query attention
07e555c
unverified

Alex Birch commited on

apply gradient checkpointing to Attention blocks
1e53ac9
unverified

Alex Birch commited on

add support for AutoModelForCausalLM#from_pretrained()'s device_map='auto'. support gradient checkpointing, probably. add lots of type hints so I could understand what's going on. multiline long method signatures/calls (for easier comparison between checkpointed/non-checkpointed variants, and because these lines got even longer when I added type hints). make MPTForCausalLM#forward accept additional kwargs, since PeftModelForCausalLM#forward tries to send it an argument inputs_embeds=None, which it didn't like too much.
9f0a20b
unverified

Alex Birch commited on

updt flash_attn_triton import (#12)
512b004

daking vchiley commited on

add flash_attn_triton.py (#9)
8f0448e

daking vchiley commited on

Update README.md
fa93455

jfrankle commited on

Update README.md
8593f0a

jacobfulano commited on

Remove print about meta init (#2)
af38791

daking commited on

Update README.md
1592fc2

abhi-mosaic commited on

Update README.md
4ca9cce

abhi-mosaic commited on

Update README.md
2412430

abhi-mosaic commited on

Update README.md
5681b9e

jacobfulano commited on

Update README.md
d7dc60a

jacobfulano commited on

Update README.md
ea6c259

jacobfulano commited on

Update README.md
2e01185

jacobfulano commited on

Update README.md
495b105

jacobfulano commited on

Update README.md
6d09da8

jacobfulano commited on

Update README.md
40d42d3

jacobfulano commited on

Create README.md
bc8d900

jacobfulano commited on

Add loss function and bool cast
c66b9f7

abhi-mosaic commited on

Upload folder using huggingface_hub
36b0251

sam-mosaic commited on

initial commit
429da0c

sam-mosaic commited on