Welcome to TorchMix!

torchmix is a collection of pre-made PyTorch components designed to simplify model development process, primarily for transformers. Our goal is to enable easy adoption of cutting-edge technologies with minimal code and maximum scalability.

One of the key priorities of TorchMix is reproducibility. All components offered by us, as well as those you create, will have the ability to self-document and can always be serialized into their hydra (opens in a new tab) compatible configurations. See Component class section for more details.

We've designed TorchMix to be as user-friendly as possible. Each implementation is kept minimal and easy to understand, using einops (opens in a new tab) to avoid confusing tensor manipulation (such as permute, transpose, and reshape) and jaxtyping (opens in a new tab) to clearly document the shapes of the input and output tensors. This means that you can use TorchMix with confidence, knowing that the components you're working with are clean and reliable.