notes

welcome to my new personal website! it's been a fun project to work on. this is my first time using angular, and i'm really enjoying it so far. i too k some inspiration from swiss web design, which is known for its grid systems, typography, and minimal aesthetic. i hope you like it!

transformers have long relied on normalization layers like LayerNorm to stabilize training and improve performance. however, a groundbreaking study challenges this paradigm by demonstrating that transformers can achieve equal or better results without any normalization layers. the key lies in a surprisingly simple operation called dynamic tanh (DyT), which re-imagines how neural networks process information.

read more on notion

large language models (LLMs) are powerful but have inherent limitations like hallucination and lacking up-to-date knowledge. to mitigate these issues, two popular approaches have emerged: Cache-Augmented Generation (CAG) and Retrieval-Augmented Generation (RAG). while both enhance LLM performance, they serve different use cases and have distinct advantages and trade-offs.

read more on notion

typography has always intrigued me. from the subtle curves of a well-designed serif font to the efficiency of monospaced typefaces, i've found myself captivated by the world of fonts and their unique personalities. my interest in design and typography led me to take a coursera course on the subject, where i delved deeper into the technicalities of type design, kerning, ligatures, and the nuanced aesthetics that make a typeface stand out.

i worked on a small program repair experiment that asked models to do two related jobs: fix buggy introductory python and explain the mistake without handing over the finished patch.

that second part is harder to score. a hint can be correct and still be unhelpful if it reveals too much. it made me think of concealment as part of answer quality, not a missing detail.

a few friends took me for meditation sessions recently, and it was a really interesting experience. i've always been skeptical about meditation and its benefits, but i was surprised by how calming and refreshing it felt.

my friends and i have been regularly playing a fun multiplayer game called 'Psych' - it's a game where you make up fake answers to real trivia questions and try to guess the correct one among the fakes. it's a blast, and we've been playing it a lot. so we decided to build our own version with country specific trivia questions, and it's been a really fun project to work on together. check it out here: Hukka Bukka (click here) - it's still a work in progress, but we're excited about how it's coming along!

using firebase's realtime database for game state management, and next.js for the frontend.

for a molecular prediction project, i tested which external examples were worth adding while only updating a small part of MolFormer. data selection and parameter-efficient fine-tuning are often treated as separate choices, but this experiment put them in the same matrix.

that made the trade-off visible: which examples enter the dataset matters alongside which parameters are allowed to move.

i redesigned the homepage by replacing a stack of little info windows with one larger statement and a slow ribbon of topics. it felt more like an introduction and less like a form someone had filled out.

there is still plenty on the rest of the site. the homepage does not need to carry all of it.

i added a little view counter to the bottom of my website — it shows the total number of page visits since i set it up.

the tricky part is that this site is hosted on github pages, which is purely static. there's no server to store state, so i needed an external service. i went with firebase realtime database (free spark plan) — each page navigation fires an atomic runTransaction to increment a counter, and onValue keeps the displayed number in sync in real-time.

the security rules are set to only allow incrementing by exactly 1 per write, so the count can't be reset or inflated arbitrarily. the number you see in the footer is the real thing.

i built a physarum slime mold simulation rendered entirely in ascii characters, and added it to the site as a background decoration on the contact page.

the simulation models physarum polycephalum - the single-celled organism famous for solving mazes. agents follow a simple rule: sense pheromone trails ahead, turn toward the strongest signal, deposit more trail. from those few rules, a branching vascular network emerges.

the tricky part was making it feel fast without blocking the page. a static poster is pre-rendered at build time (deterministic seed, 500 sim steps), served as a .txt file, and painted instantly at LCP. the live engine only loads when the element enters the viewport, fast-forwards to match the poster frame, then resumes seamlessly, so the animation looks continuous even though the heavy work was done ahead of time.

the character ramp ( \`^@/ ..„v0`) gives it that alternating diagonal texture. gamma remapping boosts faint trails so the low-density edges don't just disappear into whitespace.

i ran a set of small probes that asked a model for the average, the ideal, and then an unqualified answer to the same question. sometimes the last answer moved toward what the model had called ideal, but the effect was mixed and often disappeared.

that messiness was the interesting result. wording, language, and persona could change the direction, so a neat story about one universal bias did not survive the experiments.

i replaced google analytics with umami. page views and the few custom events i use still work, while the old analytics package and route tracking code are gone.

for a small personal site, that feels like enough.

the site moved from github pages to cloudflare pages in june. a few days later i added prerendering, so every route now ships as static HTML instead of depending on the browser to assemble the first useful page.

the site still behaves like an Angular app after load. it is just easier for crawlers and link previews to read before that happens.

rag pipelines are easy to tweak and surprisingly hard to compare. change the chunker, retriever, prompt, or model, then a few answers look better and a few look worse.

while building ProofRAG, i wanted each comparison to leave evidence behind. retrieval and answer quality get separate scores, with a report that can be reviewed later. the same run can also catch regressions before a change lands.

i have been building Tether, a local-first Android reader for articles, feeds, PDFs, and EPUBs. the hard part has been keeping one reading position between a regular document view and RSVP. switching views should not mean hunting for the same sentence again.

that constraint has shaped more of the app than i expected. bookmarks, search, highlights, notes, and each reading layout all need to agree on where you are.

i went through the project pages and gave the plots a shared visual language: the same type, quieter surfaces, and fewer boxes inside boxes. the data did not change, but the pages stopped feeling like a website with charts pasted in from somewhere else.

i like charts more when they behave like part of the article, not an interruption.

i started The Body Keeps the Score in july and am still reading it. i do not have finished thoughts on it yet.

i spent part of august testing what changes when a language model's logits stay fixed but the decoder does not. greedy, temperature, top-k, top-p, and min-p can turn the same logits into different action policies.

it made me less comfortable with treating every sampled action as a property of the model alone. the decoder belongs in the evaluation too.

i started The Heartfulness Way by Daaji in august. i am reading it alongside The Body Keeps the Score instead of waiting to finish one book before starting another.

SamplerScope now has a project page on Apart Research. the page is public, although review and feedback are still pending.

for now, i am noting the publication without calling the review complete.

every Fleet Foxes song has been a banger for me, but Shore is a bit special. it has a deeper meaning and some connections for me that the others don't, and it's my favorite album right now.

i came across Gorillaz around 2020 with Cracker Island, and New Gold had both them and Tame Impala on it. On Melancholy Hill, Plastic Beach, She's My Collar, 19-2000, Happy Dictator, Empire Ants, Tranz, Orange County... most of these have been my favorites at some point.

honestly, this one is hard to describe with just words. every Fleet Foxes song has been a banger - Mykonos, this whole album, everything. just give it a listen!

found this one through a youtube recommendation - it was a video from Zimmer90 themselves, playing in their (really beautiful) studio. the songs were really nice and gave off a summer/spring vibe.

i came across Tame Impala in 2020 with Borderline, around the same time as Gorillaz, and New Gold had both of them. Let It Happen, The Less I Know the Better, Reality in Motion and Yes I'm Changing have been some of my favorites since.

an accidental find! i was looking for background music for a video editing course i took during my bachelors, came across For Lovers and ended up liking the entire album.

i added JSONL and NDJSON support to the table preview in Zed - each line shows up as a row, and a missing field looks different from a null. if a line doesn't parse, the error points back to the line it came from.

i also fixed a crash when deleting rows in the CSV preview while i was at it. the pull request got merged!

the reviews for SamplerScope from the Apart Research sprint are in. one reviewer liked the traces and controls, but asked whether the effect comes from the decoder or just from small checkpoints answering multiple-choice prompts. the other agreed that preference studies should control for the decoder.

both are fair points - the first one is going to be harder to answer.

ProofRAG 0.9.0 is out! judging is typed now, with probabilities and a fingerprint of the rubric each run used, so two runs can actually be compared.

score gates use full precision now, and a failed A/B judgment gets reported on its own instead of quietly counting as a tie.

i rebuilt this page! the notes used to sit in one long filing cabinet, now each kind of note gets its own thing - a terminal, a flap board, a bookshelf, a row of records.

the cat on top of the ai cabinet took the longest to get right (it was an axolotl, a slime, a mushroom and a hedgehog before that).

google play granted Tether (works.tendril.tether) production access! that's the step that lets an app go out to everyone on the play store.

next up is the production release itself. more about tether

Tether 1.0.0 is out! anyone on android 9 or newer can get it on google play now.

no accounts, ads or analytics. if something feels off, the feedback screen in settings comes straight to me. more about tether