Greetings!

About me

I'm a Data Scientist and computer vision engineer based in Bangalore, India, where I am currently working with search and recommendations team at Nykaa.

Previously, I worked as a Machine Learning Engineer at Capillary Technologies Pvt Ltd. part of Smart Store+ division. Earlier to this, I was a Computer Vision Intern at Capillary Technologies and at an early stage startup SDTECH (Formely known as Studio Diseno).

I also maintain a blog "Machine Learning Cognitio" to document my understanding of computer vision and machine learning while sharing my knowledge with others.

 

Skills

Below are my skillset, and I'am always looking to learn more.

Computer Vision

I have signifcant research and industrial experience in developing Computer Vision applications using tools like OpenCV, scikit-geometery and scikit-image in C++ and Python. The applications tested my knowledge on tradional computer vision, image processing and computational geometery.

Machine Learning

My typical day at work include developing machine learning applications using TensorFlow, Keras and scikit-learn. Furthermore, I have performed data manipulation, data analysis and visulization using tools like pandas, matplotlib and seaborn. I also have experience of using PySpark for distributed data processing.

Python

Over the past five and a half years, I've extensively worked with Python in the course of research, classwork, personal projects and at work. The Machine Learning applications developed by me in Python have used Object Oriented Programming (OOPs) as it's paradigm and typing as support for type hints.

C++

My experience in C++ comes from developing computer vision applications for edge devices using OpenCV and practising competitive programming on platforms such as LeetCode, Hackerrank, Hackearth and InterviewBit. I have also compiled and generated libraries using cmake.

CUDA Python & C++

I have developed High Performance Compute (HPC) applications using CUDA C++ API to perform high speed and complex feature extraction from images to train ML algorithms. My notable work in HPC is to bring down the time required for extracting features from 1 million images to 2 hours from 11 hours on a single 1060 GTX GPU. Furthermore, I also have experinece in using Numba & OpenMP.

TensorFlow

I have significant experience using TensorFlow for training deep neural networks (DNNs) for computer vision applications. While working for Capillary, I have built custom layers, used quantization and pruned DNNs for reducing the model complexity before deploying the model on edge devices.

AirFlow

I have used AirFlow as an Orchestration tool for MLOps to automate and managing workflows such as data validation before training an AI model, training the machine learning model at specific intervals, batch processing, and inference.


  • Feb 2022 - Present

    Data Scientist

    Nykaa

    Technologies Used:

    Python, PySpark, PyTorch and Pandas

  • April 2021 - Feb 2022

    ML Engineer - II

    Capillary Technologies Pvt Ltd.

    Work done so far:

    • Automated the generation of fundamental matrix using facial key points as it's correspondence coordinates between pair of images. Each image in a pair is taken by a camera having different prespective views. Reduced the camera calibration time from 1 hour to approx ~ 5 mins for an Edge AI product (Customer Visual Profile).
    • Developed a segmentation algorithm using OpenPose (Deep Neural Network) and Watershed algorithm to create segmentation maps to segment people in an image for edge computing.
    • Developed & architected Data Validation Framework (DVF) for the product Artificial Intelligence Retail Assistant (AiRA) which is a core orchestration service logic based on Directed Acyclic Graph (DAG) to schedule data validation notebooks hosted at DataBricks using AirFlow.

    Responsibilities:

    • Lead the research & development of the POC for computer vision applications in SmartStore+ team.
    • Mentoring interns and Junior Machine Learning Engineers in their progress and overlooking the work distribution.
    • Ownership of maintaining and enhancing the data validation framework which is used to validate data before training machine learning models.

    Technologies Used:

    TensorFlow, AirFlow, OpenCV, Python, PySpark

  • June 2019 - March 2021

    ML Engineer - I

    Capillary Technologies Pvt Ltd.

    Work Done:

    • Developed an AI-product to calculate the customer's time spent (or dwell time) in a store using two modules: person re-identification and embedding matching algorithms.
      • The person re-identification is a deep learning model developed using ResNet-50 backend and a triplet loss objective function to generate an representive embedding for a person while entering the store and exiting the store.
      • The matching of embedding generated using ResNet 50 model was done using Hungarian assignment as well as K-Nearest Neighbors.
      • The model was served using TensorFlow serving on Amazon EIA.
    • Developed two versions of badge classification model using deep learning models with MobileNetV1 and MobileNetV2 backend. The model selection was done using ROC and AUC metric.
    • Developed face recognition model with different deep learning backends namely ResNet101, MobileNetV2 and ResNet50 with arc-face as it's loss function.
    • Performed post training quantization and quantization aware training on the above mentioned models to inference on edge devices such as Raspberry Pi, Google Coral Dev board and Google Coral USB Accelerator using TensorFlow.
    • Used Epipolar Geometery to assign the same visitor ID (UUID) to a customer detected in two cameras having two different prespective views. Furthermore, used Intersection Over Union (IOU) for eliminating occluded customers on the edge device.
    • Developed a document detection algorithm to detect a document in an image for an OCR product. The document detection algorithm obtained 0.93 as it's Jaccard Index.
    • Performed Exploratory Data Analysis (EDA) on client's customer behaviour towards the loyalty program provided by Capillary Technologies Pvt Ltd.

    Responsibilities:

    • Mentoring interns and overlooking the work distribution.
    • Ownership of three AI micro-services and two device modules.

    Technologies Used:

    TensorFlow, OpenCV, Python, Scikit-learn, Scipy

  • January 2019 - May 2019

    Computer Vision Intern

    Capillary Technologies Pvt Ltd.

    Work Done:

    • Developed Revolver YOLO: Person detection algorithm for fish-eye lenses. Traditional off the shelf deep learning models don't detect people in an image which have been distorted i.e. roated due to FOV being 360 degree or 280 degree. To counter this we decided to rotate (using rotational matrix) equally intrevaled patches of the image and make it upright to perform detection and rotate it back to it's original position.
    • Developed a logic similar to background subtraction to discard mannequins detected as person by YOLO.
    • Developed YOLO-Lite in TensorFlow from scratch and trained the model using an in-house dataset for person detection. The model performance was 21 mAP for the test set and gave 5 FPS on Raspberry Pi 3 B+.

    Technologies Used:

    OpenCV, TensorFlow, Python, C++

Research Papers

  1. EarNet: Biometric Embeddings for End to End Person Authentication System Using Transient Evoked Otoacoustic Emission Signal

    Journal: Neural Processing Letters | Springer | Status: Paper Accepted

    Transient Evoked Otoacoustic Emissions (TEOAE) are a class of otoacoustic emissions that are generated by the cochlea in response to an external stimulus. The TEOAE signals exhibit characteristics unique to an individual, and are therefore considered as a potential biometric modality. Unlike conventional modalities, TEOAE is immune to replay and falsification attacks due to its implicit liveliness detection feature. In this paper, we propose an efficient deep neural network architecture, EarNet, to learn the appropriate filters for non-stationary (TEOAE) signals, which can reveal individual uniqueness and long-term reproducibility. EarNet is inspired by Google’s FaceNet. Furthermore, the embeddings generated by EarNet, in the Euclidean space, are such that they reduce intrasubject variability while capturing inter-subject variability, as visualized using t-SNE. The embeddings from EarNet are used for identification and verification tasks. The K-Nearest Neighbour classifier gives identification accuracies of 99.21% and 99.42% for the left and right ear, respectively, which are highest among the machine learning algorithms explored in this work. The verification using Pearson correlation on the embeddings performs with an EER of 0.581% and 0.057% for the left and right ear, respectively, scoring better than all other techniques. Fusion strategy yields an improved identification accuracy of 99.92%. The embeddings generalize well on subjects that are not part of the training, and hence EarNet is scalable on any new larger dataset.

  2. H. U. B - Eye, Hearing using Bone Conduction and seeing through DNN

    Conference: International Conference on Advanced Computing and Communications (ADCOM 2018)

    The paper proposes a device capable of making the lives of visually impaired easier. The device encompasses an image recognition using deep learning (convolutional neural network) unit coupled with the novel idea of the bone conduction system, which can be mounted on the sunglasses of the visually impaired. The whole process allows two-channel hearing enabling people to hear regular as well as the intended audio. Instead of air, the sound is propagated through the bone in the form of vibration and is sent to cochlea through a membrane. The proposed system takes an image from a mounted camera, classifies it with a dedicated processor and sends the audio signal through a Bluetooth channel to the bone conduction transducer so that the user can hear through the system what is in front of him. The system is able to recognize the input image using deep learning and give an audio output directly to the eardrum of the user.

Education

Vellore Institute of Technology, Chennai

MTech Integrated in Software Engineering
(5 years program)
CGPA: 9.32/10.0

My education in Software Engineering focused on strong software engineering principles such as SDLC, Software Requirement Specificatin (SRS), Requirement Specification Document etc. to good programming practices in Python, C++, JAVA, .NET and Computer Networks. Furthermore, we had vigrous courses on Data Structures & Algorithms, Linear Algebra, Probability & Statistics and Calculus.

Activities

Was part of Technocrats Robotics Team for two years which participated in various robotics competitions across the nation. This team had approximately 25 students from Mechanical, Electrical & Electronics, Computer Science and Management. I was heading the computer science team for the year 2017 and all the divisions in 2018. My core contributions are given below:

  • Programmed IMU sensor to control the yaw, pitch, roll of the bot using a joystick interfaced with Arduino board.
  • Programmed a robot having LSA08 sensor for line following using image processing algorithm in OpenCV on BeagleBone Black Board.
  • Developed an algorithm for controlling the RPM of the DC motor using rotary encoder of the bot.

Achievements

A few of my notable achievements are given below:

Meritorious Award

I achieved rank 6th in the class of 221 students for the batch of 2014-2019 M.Tech Software Engineering (5 years Integrated) program with the CGPA of 9.32/10.0. I was honoured to receive the award from Ms. Smriti Irani, Honourable Minister of Women and Child Development, Government of India.

Ist place in Data Science Hackathon

Me and my teammate won 1st place among 73 teams in the Data Science Hackathon conducted by Vellore Institutue of Technology, Chennai. The competition was to predict the taxi fare for a given dataset. We built am XGBoost based model to arrive at the lowest RMSE in the competition.

IIIrd place in IOT Makeathon

Ideated and developed the product "Just in Time" which is an IOT based product for real time accident detection and notifying the helping authorities. The accident detection was done using real data collected from accelerometer and then classifying them using Dynamic Time Wrapping (DTW) Algorithm.

Best Project Award

Developed a Game in C++ using the library graphics.h in first semester for the course "Programming in C++". I had developed a carrom coin matching game for a single player.

Hobbies

Tennis

I had a liking for Tennis since elementary school days. Knocking tennis balls on the walls inside the house was my favourite passtime. In the process broke few of mom's favourite crockery, followed by being reprimanded for the same (But, I never stopped though! 😜) . My Tennis idol growing up and still date has been the one and only Roger Federer. I played Tennis regularly during my ears in school and University. Even now playing tennis is my main recreational activity. I have won till date two minor competitions in Tennis, at school level.

Chess

Being a Chess player's son (yep, my dad, Varugeese Koshy is an Iternational Master (IM) with a highest ever FIDE Rating of 2430). I learnt the basics of chess from my father although I never developed a passion for it like Tennis. I Played many rated open tournaments and ended up getting a FIDE rating of 1335. These day's I don't play open tournaments but I do play Chess Online time to time. For Accomplishments in Chess, I had decent placings in school zonals and some state level competitions.