sankalp's blog

Last Updated: May, 2025

I am Sankalp. I am a Applied AI/Backend engineer guy. At the moment, I am consulting a startup on evals for voice agents and improving the overall product. I am interested in AI engineering, LLM evals, post-training stuff and the new research in general. Open to consulting work around same. Any leads would be appreciated. Please DM me on X.**

Prior to this, I was working as a software engineer (backend mainly) at a US based fintech company for the past 2 years. My interests span AI, distributed systems, and the intersection of both. If we are being very specific, my favourite areas are AI dev tools, agentic code-generation applications.

I try to maintain a broad awareness of research trends while also focusing on applied AI stuff. I like to write, shitpost, occasionally watch anime, table tennis and playing with latest models lately. sometimes i do cinema commentary. my favourite band is Men I Trust.

Projects

This section showcases some recent side projects I built for fun and learning.

CodeQA

CodeQA is an attempt to build Cursor's @codebase feature. It allows users to search the codebase and get relevant files, code snippets by asking questions in English (natural language). It can support multiple languages.

It utilizes tree-sitter to parse AST and construct a codebase index before generating embeddings. We do a top-K RAG (Retrieval Augmented Generation) using these embeddings along with more post-retrieval techniques. I wrote a couple of blogposts (that are listed in the above section) for CodeQA.

codeQA

Demo

Note: CodeQA works 2.5x faster than what's shown in the video now. I updated it later. Check this branch

AutoLatex

Autolatex (400+ users so far)

AutoLatex is a browser extension that simplifies LaTeX equation generation for researchers and students. It uses LLMs to convert natural language and images into markdown and LaTeX, with instant rendering so you can edit on the spot. The tool offers customizable settings, keyboard shortcuts, and privacy control through user-managed API keys. By streamlining the process of incorporating complex equations into documents, AutoLatex enhances productivity in academic and research workflows, especially in deep learning fields.

VimSonnet

VimSonnet was a small project where I tried to control my browser using Claude 3.5 Sonnet as agents. Github

The approach in the demo uses Vimium (a Chrome extension that generates hint tags for all UI elements in a browser) along with Pyautogui and Claude 3.5 Sonnet as the LLM.

I had tried a screen coodinate based approach but Claude's Vision cannot point out coordinates by itself so it was required to use hint tags.

Twitter Circle

GitHub Repository

A tool to visualize your Twitter network and direct messaging history.

  1. Make a Twitter Circle visualization for up to 200 users.
  2. Check leaderboard based on combined weights of all your mentions of other users and all direct messages.
  3. Check DM stats message count per recipient, messages sent/received per user, total messages, last message with them
  4. DM bar graph where you can see messages/month for 5 years data.

This project has been used for 200+ people by now and recieved 100+ stars.

GitHub Repository

SemanTweet Search allows you to search over all your tweets from the Twitter archive using semantic similarity. A demo is available here.

It preprocesses your tweets, generates embeddings using OpenAI's small/large embedding model, stores the data and embeddings in LanceDB vector db, and provides a web interface to search and view the results.

You can do semantic search post pre-filtering by time, likes, retweets, media only or link only tweets too.

Pre-filtering by sql operations helps not only filter but also reduce the vector search space thus speeding up the search.

You can additionally use/edit projector.py and tensorflow projector to get a visualization of your tweets using t-sne algorithm as shown here

Demo Thread Visualization Demo

Handpicked by Haiku

GitHub Repository

A LLM based app that generates personalized recommendations and visualizations based on user input, leveraging Claude Haiku's knowledge base.

CaptionBot

GitHub Repository

This is my Pytorch implementation of the paper "Show, Attend and Tell".

It generates descriptive image captions for your images. Architecture involves Seq2Seq-based image captioning with attention mechanism, utilizing ResNet-50 as encoder to extract meaningful image features and an LSTM-based decoder with soft-attention and Beam Search. It achieved BLEU-1 score of 59.2 and BLEU-4 score of 19.56, closely matching the original benchmarks