Quantization

Weights are represented by numbers in a model. A normal model might store each weight using 16 bits, i.e FP16 (half-precision floating-point). Quantization reduces that dramatically, where weights are represented using INT8 (8-bit integer) or INT4 (4-bit integer). They reduce precision but increase inference speed and reduce hardware requirements. Most of the time, quantization is performed post-training, which reduces model accuracy. But there is INT4 QAT (4-bit Integer Quantization-Aware Training), which does this during training, thereby reducing the extreme accuracy drops common with standard 4-bit quantization.

1 bit weight

1-bit goes one step further and asks: what if the weights were just one bit, and if that were done during training? With exactly 1 bit per weight, each weight can have only 2 possible values. 0 or 1. 0 represents -1 and 1 represents +1. i.e., instead of storing numbers like 0.72, -0.13, 1.24, -0.56, we will store +1, -1, +1, -1. That is at least 16 times less storage for weights theoretically.

1.58 bit weight

In 1.58-bit weights (ternary bit weights), there are three possible values: {-1, 0, +1}. Theoretically, we need log2(3) bits of information to distinguish between 3 possible states : 2^x=3. i.e. log₂​(3)≈1.585. Hence the name.

For our interpretation and understanding:

• 0: the input is effectively ignored for that connection. i.e., no influence at all, which also gives you built-in sparsity.
• +1: the input contributes positively to the output. i.e., an increase in the input pushes the weighted sum up. This is “same-direction” influence.
• -1: the input contributes negatively. i.e., an increase in the input pushes the weighted sum down. This is “opposite-direction” influence.

1.125-bit weight

1.125-bit applies different logic for compression and calculation. One can’t literally allocate 1.125 bits per weight. The way this works is that the average bit allocation is used when they are packed. For example, it takes 9 bits to store 8 weights; thus, the average number of bits per weight is 1.125. One important distinction here is that bits per weight doesn’t necessarily tell you how many values each individual weight can take. 1.58 has a clean theoretical connection to three states, while 1.125 bpw usually describes the overall encoding/storage scheme. The table below should make a bit clear

BitNet b1.58 2B4T

This is the first open-source, native 1-bit Large Language Model (LLM) at the 2-billion parameter scale, developed by Microsoft Research. You can find the model on the hugging face, it needs a specific inference code called bitnet.cpp.

Bonsai models

Bonsai 1-bit models are probably the first set of production-ready binary/ternary models. They have 1.7B, 4B, 8B, and 27B parameter models.

I have been running the 8B-parameter version locally. My experience with it has been positive. I have explored it for text summaries, code explanations, etc. I will try to install the 27B version and do some coding to see how it goes.

# Install llama
curl -LsSf https://llama.app/install.sh | sh
# Serve the model using OpenAI-compatible API and a web ui
llama serve -hf prism-ml/Bonsai-1.7B-gguf:Q1_0
# or 8B parameter version
llama serve -hf prism-ml/Bonsai-8B-gguf:Q1_0

Note: Yes, I have moved away from Ollama and now mostly use llama.cpp. Also I sometimes Jan.ai along with OpenWebUI

• I played host for them; actually, a few team members did all the work. I was just present, that’s all. That said, that kept me busy during the week. We booked a hotel on Church Street through a popular booking site, but they canceled at the last minute. Then we had to book the same hotel using a different aggregator site. This is the second time for me. So I think from now on, after booking a hotel online, I will call the hotel to confirm. The confirmation from the aggregator means nothing.
• We had team lunch at Brik Oven and team dinner at Ishaara, both on Church Street. We also took some time out together to spend a day at Our Native Village near Hesaraghatta.
• I ran a bit on Sunday; my calf muscles were hurting, so I stopped. But I was able to run on Monday and Tuesday at Hesaraghatta grasslands. It’s so different to run in the green surroundings compared to running in the city. I could do only one P.T session and one game.
• On Sunday, I went to watch the Bengaluru’47 exhibition at RBNAM School. I loved it.
• I wrote a simple web component called muscle-map to log the workouts here.
• I have lots of image albums on my phone which are collections on random subjects. One of them is titled “Unidentified Urban Objects” in Bengaluru. This week I took an opportunity to upload to an Arena channel called Unidentified Urban Objects. It has lots of random pictures of manholes, electric poles, public sensors, toilets, and gods on the footpath, etc. You can also add to the channel. All Bangalore-focused.
• I am probably one of the few paid Grammarly users who still use classic mode to avoid AI rewriting in their new interface. Also, the new interface is not writer-friendly; too many distractions. 1-bit, 1.125-bit, and 1.58-bit LLMs are very interesting and are capable. I have started exploring some of them and will probably productize some.

After that, I wanted an easy way to embed it on any web page. I chose to use web components. It converts the given list of exercises into muscle groups, then applies styles to them within the SVG and embeds the SVG. I have verified it to the best of my knowledge and from available sources. It took a lot of battling with Claude to make it the way I want, that is, to make it generic and reusable. I still have to map a bunch of muscle groups, I have listed all the short comings in the readme.

Finally, I created a WordPress shortcode to queue the JavaScript and embed the component. But that’s not really required. If you have a simple static site, then you can include SVG and a JS file and use the muscle-map tag in your HTML

Wednesday Workout

Run

Calf raise, Tibialis raise

Calf raise

Happy workout and happy logging.

• I have been using Tau by HF as the coding agent. It’s very simple but has been very good. It’s just enough. This one is different from Tau by dpc, which also looks good. I am yet to try it. It’s on my list to try. Work in general has been good, and I am happy about the progress I have been making.
• They have started digging everywhere in Bangalore. Almost all the roads around our home have been dug up. I can see the work is getting done, but I don’t know why they had to do it on all roads at once. Now it’s an adventure ride to Uma’s school every day.
• These days, every Tuesday is a cupcake Tuesday for Uma. We stop at Cafe Da Costa Bakery on our way back from school.
• Saturday has become the rest day. Originally, I wanted to do a longer run on Saturdays, but it has not happened yet. Sunday was also mostly a rest day as it rained when I started running. Bangalore rains are unpredictable. So I immediately stopped and returned home. As usual, I did two P.T sessions during the week. I also started logging the main part of the workout on Strava for later reference and to measure progress. I also have a reliable partner to play now, so we played two games. I couldn’t run today as I was driving back home :( I do recognize that I can dedicate an hour each day to physical activity because I work mostly from home and save my commute time. A couple of months back, I got a pull-up bar for a door frame. Until now, it has been used mostly by Uma and me for dead hangs. This week I was able to do two proper pull-ups at home. I have been practicing assisted pull-ups during P.Ts, so it’s not a surprise. That said, I have not yet reached the goal (WN 23/2026) of 3 proper pull-ups.
• I had an interesting conversation with Abhishek from OpenConsultancy about Frappe and related products. I know about Frappe; I have used their libraries but not their main product. But it was good to catch up with Abhishek and learn more from a fellow FOSS enthusiast.
• Tomorrow is a holiday, so only one class this week. I left at the wrong time today. At 5pm and it took me 3 hours to reach home :( that’s why no run.
• Appa was here over the weekend for his dental treatment. My dental treatment is almost done! Maybe two more months :(
• I have this book, Coders at Work. I was wondering: with AI and agents, does it still make sense? It makes more sense than ever. Get a copy or borrow from me. I have a hard copy. You would have seen GPX/Map animation videos in my previous blog posts; those are created using a FOSS tool called GPX Animator. It has lots of features to explore.

My trail run training is progressing okay. It’s not spectacular, but I am on it, which is what I think matters. I am nervously excited about it. I am inspired by Soham’s posts about his trail runs. Hopefully, I get to run and complete in Coorg.

• I met a bunch of IITM/Study students at IISc at a students’ gathering on Saturday. It was fun to meet them and have a conversation. It was drizzling a bit, and IISc was so pretty. Nothing can replace the trees.
• I am teaching World of Computing this term at APU. The Sem/Term has started. So most Fridays and Saturdays I am working from the campus. It’s also a beautiful campus to run.
• Both Uma and Anju have been down with a cold this week. There is also a lot of road repair (digging) work that’s going on around us. It has become very dusty. It has contributed a lot too.
• License to Learn is a project by FOSSUnited on Indian academia’s expenditure on non-free software. Currently, it has data from NITs. Hopefully, in the future it will have data from IITs and other big universities like VTU. This reminds me of my failed effort almost two decades back, pushing VTU to use FOSS. FOSSUnited is offering a scholarship for FOSS developers to pursue the Graduate Certificate in Public Policy (Technology and Policy at Takshashila Institution. Do apply. ConstitutionOfIndia.net is a good site for educational material on the Constitution of India, including the Constituent Assembly discussions. I like the fact that the whole Constitution is available as an HTML document.
• The Geological Survey of India (GSI) has published village boundaries for all the villages in India. They have published them in both KML and GeoJSON formats. I have downloaded it. I am yet to upload it to the archive.

I tried running again a few years back and dropped it. I wanted to start running again. The best part of running is that one can do it alone or with company, unlike many other games. Last week, I started running. I was not sure how I was going to feel and how my body was going to react. So I didn’t stress much about it. I started slowly and paced myself, giving time to adjust.

This also means I have to slot this between my workout days. I have fixed P.T. days; two days out of Mon, Wed, and Fri. Most weeks it’s Mon and Wed. So runs are going to be either Thu, Fri, or Sat. Though my initial plan was to run a week three times, I will be happy with two days.

I have this grand plan of running The Coffee Trails 11KM in Coorg this Octbober. That is if everything goes well. It all depends on how the first few weeks go. It seems possible now based on Week 1. Fingers crossed.

• As long as Caste exists, reservation must exist. Even after reservations, big institutes have institutionalized caste-based discrimination; we have seen many cases of that. The rot is deep. So if you think caste-based reservation has to go, then you work on the root cause. Annihilation of Caste is the only way.
• I have five or six Indian UNCLES (Some are five years younger than me and others are at least ten years older than me) on Twitter. They are very sweet and agreeable when you meet or talk to them. But full of venom on Twitter, once in a while I check their Twitter posts to see what troubles them. Gen Z seems to be troubling them a lot these days.
• I wrote a blog post about my ride to Gandikota and the struggles. I am yet to test ride (long, like 100+KM) a KTM AdvX.
• I continued with P.T, and no games. I had to skip today’s P.T because of an emergency. I have restarted running. This week I ran thrice. I ran today because I skipped P.T. Otherwise it would have been only twice. But the plan is to run three times a week if possible. I capture the path using GPSLogger and send it to my CouchDB. I am also logging it to Strava to find community, especially as I want to run a 10K in October. Hopefully I will be ready. I got myself a pair of Nike Stride Aero Fit. Usually all shorts give me chafing. But this one has been great. I do own Puma ForeverRun NITRO; it’s a good pair of running shoes that I have not used much. Sometimes you buy things in enthusiasm but never use them. This was one such buy. Just after that buy, I had neck issues, and I couldn’t jump or run much. Now at least they will get used. Also my 12 year old jersey from 2014 Tata 10K still works and looks great.
• I have been using Impakto Barefoot Shoes to walk around and also use them inside the gym. Impacto has been a surprise; they have been very comfortable. They are affordable and very lightweight to carry. I will have to write a detailed review about them.
• I registered myself at the Independent Director’s Databank. I am yet to go through their online course, but if you are looking for one, you can find me there.
• I really like this CheapFoodMap. We have enough data for IDVC to build something similar. I used Claude to build it. It’s hosted with the data, and it’s live. Let me know what you think. You can contribute data to IDVC using a simple form. I review and approve it before it goes live. I love Pocket Bird. It’s a small bird that lives inside your browser that you can interact with. You can set it up on your site for your users to play, or you can install the add-on so you can play on any site.

That worked well; there was very little traffic on the road. What didn’t work out was my plan to ride the KTM 390 Adventure X (2025) model. I couldn’t get an acceptable bike yesterday. So I had to settle for a Suzuki V-Strom SX (250cc) instead of canceling the whole ride. That’s the closest I could get.

The V-Strom is lighter but taller at 835 mm vs. 825 mm of the AdvX. But the AdvX is heavier at 181 kg vs 167 kg. They are both slim. On the V-Strom, my toe tips would touch the floor if I tried both feet, but I was comfortable with one leg. I struggled initially, especially inside the city stop-and-go traffic. I also struggled when trying to do sharp U-turns from the stop. I surely can’t lift it if it falls, at least for now. But after about 300 Km, I grew confident and used some tricks like a butt-scooch stop or a side-straddle stop, climbing on it to mount, turning on the side stand, etc. These tricks made it much easier. Finished 600+ km without any incidents. There is still a lot to learn about these tall motorcycles.

The ride itself was good, except maybe a 70KM stretch which was bad with lots of under-construction roads and dust. It was also very windy, which made it difficult. But there was no traffic, and the other roads were really good. So, all in all, it was a good ride.

Gandikota itself was pretty; the gorge is spectacular. You can also walk around in the fort. It’s a good weekend drive or ride from Bangalore if you plan to. Route we took are available in GPX and GeoJSON for you to download and use.