At Encord, we're building the AI infrastructure of the future. Today, the biggest challenge companies face in getting an AI product to market is actually not half as glamorous as it may seem: it's all about data quality. In fact, the success of any AI application today relies on the quality of a model's training data — and for 95% of teams, this essential step is both the most costly and the most time-consuming.

As ex-computer scientists, physicists, and quants, we felt first-hand how the lack of tools to prepare quality training data was impeding the progress of building AI. AI today is what the early days of computing or the internet were like, where the potential of the technology is clear, but the tools and processes surrounding it are still primitive, preventing the next generation of applications. This is why we started Encord.

We’re a team of 60 working at the cutting edge of computer vision and deep learning, backed by top investors, including CRV and Y Combinator, leading industry executives like Luc Vincent, former VP of AI at Meta, and other prominent leaders in AI. We are one the fastest growing companies in our space, and consistently rated as the best tool in the market by our customers.

About Us

At Encord, we're building the AI infrastructure of the future. One of the biggest challenges AI companies face today is data quality. The success of any AI application relies heavily on the quality of its training data, yet for most teams, this crucial step is both the most costly and time-consuming. We’re here to change that.

As former computer scientists, physicists, and quants, we’ve experienced firsthand how a lack of tools to prepare quality training data impedes progress in building AI. We believe AI is at a stage similar to the early days of computing or the internet—where the potential is clear, but the surrounding tools and processes are still catching up. That's why we started Encord.

We are a talented and ambitious team of 60, working at the cutting edge of computer vision and deep learning. Backed by $30M in Series B funding from top investors like CRV and Y Combinator, we’re one of the fastest-growing companies in our space. Our platform is consistently rated the best by our customers, and we have big plans ahead. We’re looking for a Research Scientist to help our customers get the right data faster, easier, and cheaper.

The Role

As a Research Scientist focusing on multi-modal LLMs, you'll be allowing all the data, metadata, and embeddings that live in our system to be explored, used, and analyzed in ways no one thought possible. Although starting narrow with “smaller” multi-modal problems like, e.g., improving similarity searches via metadata, we have high ambitions for this role. You'll progressively work on harder problems that will improve user experience, surface the right (personalized) analytics to every customer, and put our users in the driver's seat of a data development platform that can do things much beyond today’s standards. Tasks can be i) fine-tuning models to understand how our platform is used by customers, ii) employing LLM reasoning to assist customers in their data analysis tasks, and iii) Building tools for customers to interface naturally with our platform. All to put the power in the hands of anyone using Encord.

You'll follow the latest research and accelerate state-of-the-art technologies to enrich customers’ data journeys. This role offers a great growth opportunity, with the potential to lead a bigger team of scientists over time in our efforts to build the ultimate data development platform

What you will be doing:

• Building, fine-tuning, and experimenting with multi-modal LLMs to surface potential actions and analytical conclusions in a data-driven manner.
• Developing scalable and novel ways to personalize LLMs based on information from our data development platform.
• Build sophisticated RAG systems on other types of data than the usual text documents.
• Follow the latest machine learning research to identify and apply new methods that improve our processes or the user experience.
• Ensure our customers have the world’s most powerful AI-powered data development platform.

Skills for the job:

• A PhD or similarly strong academic background in machine learning, with 2+ years of hands-on experience in with LLM fine-tuning, RAG systems, and prompt engineering.
• Proficiency with frameworks like PyTorch, Tensorflow, JAX, Pandas, and OpenCV.
• A solid understanding of transformer models and their common variants, loss functions, and pitfalls.
• A quick learner with a structured, organized approach to problem-solving.
• Excellent communication skills with an ability to uncover use cases and solve problems efficiently.
• Ambitious and self-motivated, with a proven track record of top performance in academic or professional settings.

Bonus skills:

• Experience working with data in the order of millions.
• Familiarity with using (and adapting) models like LLaMa and LLaVa.
• Experience with image-to-text embedding models like CLIP and SigLIP.
• Familiarity with cloud-based model training and inference.

What We Offer

• Competitive salary, commission, and equity in a high-growth business.

• A collaborative, in-person culture with most of the team working in the office 3+ days a week (engineers typically work on-site Wednesdays).

• 25 days annual leave + public holidays.

• An annual learning and development budget to help you grow your skills.

• Company lunches twice a week and regular socials, including bi-annual off-sites.

At Encord, you’ll have the unique opportunity to be part of a fast-growing startup with a clear mission and vision. You’ll work on real-world AI use cases across a variety of industry verticals and get hands-on experience with cutting-edge computer vision and deep learning technologies. This is a role where you'll grow quickly, take ownership of projects, and help shape the future of our company.

The role will be exposed to a broad tech stack (e.g. ReactJS, Python, REST & GraphQL, OpenCV, PyTorch, GCP, AWS & CUDA, Kubernetes) and the cutting edge of computer vision and deep learning.