Ayush Ranjan

Ayush Ranjan

Graduate Student At University of California

About Me

Hi, I’m Ayush Ranjan from Madhubani, Bihar, India. I build practical AI and backend systems- the digital “bridges” that connect products and people. I earned my B.Tech in IT from Manipal University Jaipur and I’m completing an MS in CS at UC Santa Cruz.

As a Java Backend Engineer at Capgemini, I developed and maintained enterprise-level applications for Mercedes-Benz XDIS, focusing on diagnostics systems and data analytics tools. I successfully delivered multiple software features while implementing performance optimization and system reliability improvements.

I led a micro-frontend architecture project that earned 3rd place at Innocircle 2022, Mercedes-Benz's internal innovation hackathon. Our solution achieved a 50% reduction in topology review time, demonstrating strong problem-solving skills and technical leadership.

Currently pursuing advanced studies at UC Santa Cruz, specializing in Applied Artificial Intelligence, Retrieval-Augmented Generation (RAG), agentic workflows, and web automation agents. I gained hands-on AI research experience as an AI Research Intern in the Information Retrieval & Knowledge Management Lab under Prof. Yi Zhang.

Currently serving as a Graduate Researcher in the AI Explainability & Accountability (AIEA) Lab under Prof. Leilani H. Gilpin, focusing on Retrieval-Augmented Generation(RAG) and AI ethics.

My teaching experience includes serving as a Teaching Assistant for Database Systems (four quarters) and Software Engineering courses at University of California, Santa Cruz, developing strong communication skills and technical mentoring abilities.

I specialize in Backend Development and Applied AI with strong Full-stack Development capabilities. Whether you're building scalable enterprise systems, implementing AI-driven solutions, or developing production-ready applications, I bring proven experience in shipping high-quality software solutions.

Apart from studies, I'm a big fan of football (soccer)- both as a player and a spectator, and I find pure joy in the game. Beyond the field, I'm an avid reader of non-fiction, especially on technology and its impact on our world. I'm also deeply inspired by the Indian epics, the Mahabharata and the Ramayana, which I believe hold timeless wisdom and captivating tales. When I'm not chasing a football or immersed in a good book, you can often find me cooking something delicious.

Work Experience

AI Explainability and Accountability (AIEA) Lab, UCSC

Graduate Researcher

(Oct 2024 – Current)
  • Conducted applied research to improve the reliability and explainability of LLM-based university chatbots across campus use cases (enrollment, deadlines, housing, course queries), leading to higher user satisfaction.
  • Designed and evaluated 10+ advanced RAG workflow architectures- Classic RAG, Chain-of-Thought, RARE RAG, Adaptive RAG, Corrective RAG, RAT RAG-using a comprehensive RAGAS evaluation framework.
  • Fine-tuned open-source LLMs to align with university-specific tone, structure, and factual accuracy, enabling domain adaptation for student and administrative queries.
  • Achieved consistent 35–50% performance improvement over baseline RAG systems, with approaches excelling by metric-faithfulness, answer relevancy, and context precision-based on query complexity and domain.
  • Developed a production deployment pipeline using Docker, Kubernetes, and FastAPI with automated CI/CD, load balancing, and monitoring for a scalable campus-wide chatbot implementation.

Information Retrieval and Knowledge Management Lab, UCSC

AI Research Intern - Stealth Hardware Startup

(July 2024 - September 2024)
  • Built a 0-to-1 multimodal AI agent for smart wearables (camera-integrated earphones), implementing wake word detection (WWD), intent classification, and real-time audio-visual processing for calorie estimation, emergency response, and video summarization.
  • Designed an intelligent query routing system with 95% accuracy at classifying continuous vs. new queries, integrating Dialogflow for 8+ pre-built workflows (calorie estimation, contact calling, emergency location services) and custom LangGraph agents for open-domain conversations.
  • Engineered a real-time multimodal data fusion pipeline combining audio transcription (Whisper), computer vision (food segmentation, depth estimation), and vector similarity search with intelligent fallback to external tools (web search, OCR) when confidence dropped below the 0.8 threshold.
  • Developed a multi-threaded memory manager to asynchronously encode and cache historical observations (images, transcripts) into vector embeddings using Hugging Face Transformers, with persistent storage in Pinecone.
  • Integrated the prototype with a local edge pipeline (FFmpeg, Whisper, custom CV models), achieving sub-500ms inference latency for key commands and enabling real-time calorie detection via food segmentation and depth estimation.

Capgemini Technology Services India Limited - Mumbai

Associate Consultant

(Oct 2022 - Aug 2023)

  • Headed the Data Modeling team for Mercedes-Benz’s XDIS platform, driving backend schema evolution for vehicle network topology change requests (e.g., ECU reconfigurations, bus architecture edits).
  • Designed a lightweight ETL pipeline in Java to process large XML diagnostic files-extracting telemetry, transforming into updated entity models, and loading into IBM Db2-enabling seamless data migration.
  • Authored and tuned complex SQL queries and views in Db2 for schema validation, relational consistency checks, and historical topology comparisons supporting Change Request (CR) automation.
  • Achieved 3rd Place at Innocircle 2022 by implementing a micro frontend architecture that let users modify and review vehicle network topology changes, reducing process time by 50%+.
  • Built an AI-assisted validation system for 2,500+ historical CRs using Word2Vec and Sentence-BERT embeddings of symbolic topologies, flagging rare configurations and recommending optimal topologies to improve validation accuracy.

Senior Analyst

(July 2021 - Sep 2022)

  • Initially, worked as a Java Full Stack Developer on the 'Arek Oy' project, a Finnish company, on the development of banking system. This project primarily focused on frontend development and utilized a tech stack comprising React, Spring Boot, Redux, and GitLab for version control. It followed a Maven project structure, with a MySQL database as the data repository. Responsibilities included frontend development, tech stack integration, and codebase maintenance.
  • Later, transitioned to a Java Developer role at Mercedes-Benz Research & Development India's Project XDIS (Cross-platform Data Information System), a critical tool for vehicle diagnostics and automatic driving scenarios in Mercedes. Conducted comprehensive software analysis, programming, and proficiently handled testing and debugging. Contributed to creating well-designed, efficient, and testable code that contributed to project success.
  • XDIS is a core Java program with a swing-based user interface, integrated within the vehicle to download diagnostic data as well as to assist users to change the network topology of their cars. It followed a gradle project structure, with a IBM Db2 database as the data repository.
  • Dramatically optimized XML file migration time by an impressive 66.67%. Additionally, enhanced the tool's robustness by concurrently implementing indexing strategies for associated database tables.
  • Optimized export testing by developing a wrapper around the Autosar framework and implementing an efficient XML file import strategy, reducing overall testing time by 40% and improving export performance for individual modules by an average of 17%.

Senior Analyst Intern - Capgemini Technology Services India Limited - Pune

(Jan 2021 - May 2021)

  • Worked as a Java Full Stack Developer, collaboratively engaging in both frontend and backend development.
  • Utilized React for frontend development and Java Spring Boot for backend tasks.
  • Key project involved the creation of an online medical portal, catering to four distinct user roles: User, Doctor, Nurse, and Admin.
  • Ensured thorough documentation for this innovative digital solution.
  • Seamlessly integrated the frontend and backend via Axios, enhancing user experience and data security.
  • Rigorous testing procedures were executed, employing JUnit for the backend and Jasmine for the frontend.

Key Projects

Unveiling Glitches: A Deep Dive into Image Encoding Bugs within CLIPAI

Recieved A+ in CSE 290D Neural Computation at UCSC for this project.

project name
  • The research project focuses on uncovering glitches in image encoding within CLIP, a model known for its integration of vision and language processing. By employing methodologies like the Discrepancy Analysis Framework (DAF) and the Transformative Caption Analysis for CLIP (TCAC), the study aims to evaluate CLIP's performance and identify areas for improvement.
  • The Discrepancy Analysis Framework (DAF) method is a systematic approach used to evaluate CLIP's performance by comparing its image similarity rankings with those of the DINOv2 model.
  • The Transformative Caption Analysis for CLIP (TCAC) method is an extension of the Discrepancy Analysis Framework (DAF) that focuses on evaluating CLIP's response to image transformations. This approach involves setting up various transformations to simulate real-world conditions, predicting caption probabilities before and after transformations, and manually inspecting images and captions for discrepancies.
  • Through systematic analysis, we reveal discrepancies in CLIP's interpretation of images compared to human perception, highlighting 14 systemic faults, including 4 novel faults.
  • By addressing these limitations, the study lays the groundwork for the development of more accurate image embedding models in artificial intelligence.
  • You can refer to this ppt or github repository to understand more.
You can read the research paper here

Sentiment Analysis With CNN AI

Recieved A+ ( 10/10 ) as my minor Project in my undergrad at Manipal

Supervised by: Shashank Sharma

project name
  • The project is based on implementing Convolutional Neural Networks (CNN) for sentence classification, inspired by the paper "Convolutional Neural Networks for Sentence Classification" presented at EMNLP 2014. The aim is to extract features from text using different filter sizes and numbers to learn various n-gram features.
  • The project utilizes the Spacy library for various NLP tasks such as tokenization, lemmatization, part-of-speech tagging, entity recognition, and dependency parsing. The CNN architecture is designed with multiple convolutional layers of different filter sizes (2, 3, 4, and 5) to capture bi-gram, tri-gram, 4-gram, and 5-gram features respectively. Dropout regularization is applied to prevent overfitting.
  • The dataset is split into training, validation, and test sets, with the vocabulary size limited to 50,002 words, including special tokens for padding and unknown words. Each word is represented using one-hot encoding and passed through an embedding layer to convert them into word embeddings.
  • The forward method of the model involves passing the text data through the embedding layer, followed by separate convolutional layers for each filter size. After applying the convolutional layers, the output tensor is passed through ReLU activation to introduce non-linearity. The resulting feature maps are then concatenated and passed through a linear layer for sentiment prediction. The model is trained using the Adam optimizer and BCEWithLogitsLos function, which combines sigmoid activation and binary cross-entropy loss. The initial representations of word embeddings are obtained from pre-trained GloVe embeddings.
  • After training the model for just 5 epochs, it achieves a test accuracy of 87%, a validation accuracy of 89%, and a training accuracy of 88%. Overall, the project demonstrates the effectiveness of CNNs in text classification tasks using Pytorch.
More about this Project
project name

Enhancing Image Captioning with Attention MechanismsComputer Vision NLPDeep Learning

  • Developed and implemented a baseline LSTM model with Resnet50 for feature extraction using an encoder-decoder architecture.
  • Integrated attention mechanisms to enhance the model's performance, achieving significant improvements with minimal training epochs.
  • Conducted extensive experiments, including custom data splits and benchmarks, evaluated using BLEU metrics.
  • Addressed and analyzed irregular validation loss, exploring learning rate schedules and comparing results with existing implementations.
  • Proposed future investigations into larger datasets and beam search strategies to improve inference.
  • Completed this project as part of my Advanced Computer Vision course, securing an A+ grade.

View on Github

project name

Video to Mp3 Converter Microservice Software Enginnering

  • Developed a microservices-based system with four services, including an authentication gateway, authorization service, video upload service, and converter service. The gateway authenticates users via an authorization service, generating JWT tokens for valid users, enabling secure video uploads. Video-to-MP3 conversion was facilitated using the Python library ”moviepy”.
  • Implemented asynchronous communication using RabbitMQ queues to facilitate seamless video processing and conversion to MP3, ensuring efficient task distribution among services.
  • Utilized MongoDB with GridFS for efficient storage and retrieval of large video files, overcoming MongoDB's 16MB size limit.
  • Managed video and audio file storage, handling, and conversion while ensuring data integrity and secure storage mechanisms.
  • Utilized Docker for containerization, Kubernetes for orchestration, and Minikube for local development, ensuring consistent and scalable deployment across environments.

View on Github

project name

Covid-19 Detection from CT-Scan Deep Learning

I applied 4 layered CNN architecture using Keras on CT dataset of just 350 Ct-Images of 219 peoples to identify covid and non covid patients used 4 convolutional layer followed by Max-pooling. To calculate loss, I used binary cross-entropy. Used data augmentation and dropout to tackle overfitting. Got an training accuracy of 78.9% and test accuracy of 67.3% on such a small set of data.

View on Github

project name

Facial Attendance System AI Software Enginnering

Supervised by: Ginika Mahajan

Utilizing OpenCV, the system captures image frames and employs Haar features and Cascade Classifiers to detect faces within them. Identified faces are then compared against a database for identification. Attendance records of recognized individuals, including date and time stamps, are logged, with the capability to retrieve historical data using names.

The GUI, built with Tkinter, encompasses attendance features and admin query capabilities for login times. Integration of Google Text-to-Speech enhances user interaction with personalized welcome messages for recognized individuals.

View on Github

project name

Personal Website using React Software Engineering

Designed and developed a personal website using React , Bootstrap, JavaScript, HTML and CSS. Deployed the website using Github Pages.

View on Github

Visit the Site

project name

Avoid-Obstacle Game Algorithm Design

Developed a Python(Pygame)-based interactive game featuring character movement control for left and right directions, allowing players to navigate and evade dynamically changing obstacle blocks. The game's challenge dynamically escalates as the speed of the blocks doubles with each successful evasion, providing an engaging and progressively challenging gaming experience.

View on Github

My GitHub Activity