You May Also Enjoy
Microsoft MX Format Emulation Tool on Ubuntu 22.04
1 minute read
Published:
Microscaling (MX) data formats are designed to reduce computational and storage costs in deep learning applications by combining a per-block scaling factor with narrow floating-point and integer types for individual elements. This approach balances hardware efficiency, model accuracy, and user convenience. Empirical results from popular benchmarks demonstrate the practicality of MX data formats as a drop-in replacement for standard FP32 in AI inference and training, with minimal user friction. Notably, MX formats have enabled the training of generative language models with sub-8-bit weights, activations, and gradients, incurring only minor accuracy losses without requiring modifications to the training process.
Setup NVIDIA Chat with RTX on Windows 11
1 minute read
Published:
βChat with RTXβ is an application that allows customization of a GPT large language model (LLM) with your specific content like documents and notes. This enables you to generate contextually relevant answers from a bespoke chatbot. The entire setup operates locally on RTX-enabled Windows PCs or workstations, ensuring quick and secure access to information.
GPU Related
less than 1 minute read
Published:
GPU-related notes
Remote jupyter notebook via ssh port forwarding
1 minute read
Published:
Remote jupyter notebook can be useful when the remote computer has better computing power than the local one or the remote is a server that you can connect from different places.