CLN - Feature
Jonathan Montgomery was intrigued when his manager told him about the Association for Diagnostics & Laboratory Medicine’s (ADLM’s) latest data science challenge. Called LabDocs Unlocked, the 2025 contest asked participants to build an artificial intelligence (AI)-powered tool that could accurately extract information from complex laboratory documentation stores.
Montgomery, a systems specialist with Indigo BioAutomation and master’s student in data science and AI at the University of Denver, thought that sounded like a fun project. So he dedicated his free time over the next several months toward building his program, ensuring the tool could retrieve information accurately and quickly, whittling down the time between a query entry and response to just 4.5 seconds.
His efforts paid off. Montgomery’s tool, “Contextual Retrieval for Intelligent Chatbots,” was voted the winner of the fourth annual ADLM challenge during a live demonstration webinar in February that was viewed by more than 200 people, including laboratory directors and other data analytics and informatics experts.
“I am excited about the opportunity to leverage AI to extract value from internal documents not in the public domain,” Montgomery said. “These AI tools and concepts open a new frontier to improve business efficiency, quality of life, and human-computer interactions, and I’m excited by the success of my design as a reusable tool for knowledge retrieval.”
The competition was tough, said challenge organizers Mark Zaydman, MD, PhD, an associate professor of pathology and immunology at Washington University in St. Louis, and Dustin Bunch, PhD, DABCC, FADLM, director of laboratory informatics and data analytics at Nationwide Children’s Hospital in Columbus, Ohio.
The competition required entrants to pose 15 trial questions to their interfaces and record the responses after searching for information among 13,600 PDF documents of different sizes and formats, such as standard operating procedures or Food and Drug Administration 510(k) submissions. The tools they created had to not only answer the questions correctly, but also provide a link or access to the primary source.
Montgomery’s program was one of nine submitted. Zaydman and Bunch evaluated each program independently using a scoring rubric they developed with other members of ADLM’s Data Analytics Steering Committee; Andrea Hobby, ADLM’s director of data science; and Matt Boyle, ADLM’s senior director of publishing and scientific content.
They reviewed each entry with four criteria:
Zaydman and Bunch also took into consideration the entrants’ comments and their use of version control software to track development. Then, the two organizers met with Hobby to discuss their results and select two finalists: Montgomery and a group from Baylor College of Medicine in Houston.
Next came a live webinar, in which Montgomery and pediatric neurologist Mikael Guzman Karlsson, MD, PhD, a clinical informatics fellow at Baylor College of Medicine and Texas Children’s Hospital, described their tools and what inspired them to enter the contest. Karlsson and his colleagues developed PathFinder, a generative AI tool that also uses RAG to help laboratory teams quickly find and interpret essential information buried in complex diagnostic and regulatory documents.
During the program, Zaydman and Bunch gave the finalists three new challenge prompts, such as, “What specimens are acceptable for the quetiapine assay?” Audience members watched as each tool gave its responses live and then voted on their favorite. While the finalists answered audience questions, ADLM staff tallied the votes so that the winner could be announced.
Montgomery said he was more excited than nervous leading up to the event.
“I ran through my slides a ton of times just to make sure the timing was right, because 10 minutes is not a long time,” he said. Montgomery tried to find a balance between explaining what he did on the engineering side and providing information that could be useful to audience members who were considering similar solutions. “There are some fundamental concepts that I wanted to bring people up to speed on, so if they’re looking to purchase something, they can have some idea what’s ‘in the box.’”
Although Zaydman and Bunch were impressed by all of the contest entries, Montgomery’s was a clear standout.
“First of all, it was extremely accurate,” Zaydman said. “The quality of responses it gave was exceptional. It was complete, but it was also very concise. It gave you exactly what you needed, and then it didn’t drone on and waste your time with extraneous detail. That was really appreciated.”
Montgomery put a tremendous amount of engineering into the program, Zaydman added. In the documentation he provided, he described experiments he ran to optimize the quality, speed, and accuracy with which the tool retrieved documents. “You can clearly see that he’s a professional software developer in the quality of his code,” Zaydman said. “He hit all the best practices as well, so he really knocked it out of the park across all our scoring domains.”
The submitted tools varied in approach to information retrieval, programming languages, and large language models, said Zaydman and Bunch. Even among the two finalists, the Baylor team made use of enterprise tools out of the box, while Montgomery’s platform was mostly developed from scratch. “It illustrates that both approaches can produce a really well-performing tool,” Zaydman said.
Zaydman and Bunch selected AI retrieval of documentation as the focus of this year’s contest because it’s a relatable problem that all labs struggle with, they said.
“We’re at a point where AI and data science might radically change this process in the near future,” Zaydman said. “We wanted a challenge that would incorporate large language models and generative AI. This is a technology that has really evolved rapidly in recent years, and there’s a lot of excitement around the potential, but at the same time, there are safety concerns.”
Humans face many challenging tasks in the lab that they either don’t have sufficient time for or find mundane and unenjoyable, Zaydman added.
“We have so much documentation, protocols, regulatory documents, and checklists. They change and it’s hard to keep up to date,” he said. “Even just finding where the information is stored out of the thousands of documents can take an excruciating amount of time. Our current manual process is terrible, to say the least. It’s a pain that’s felt across every laboratory.”
For example, a person might click on a document and wait 20 seconds for it to load, only to realize it’s not the one they want, he said. So they click on one in a different folder and repeat the process. Being able to access correct information quickly is essential. “If there’s friction, [people are] going to rely more on their memory than going and verifying, and that’s not the safest or best way to proceed,” Zaydman said.
Montgomery’s company is evaluating the tool’s business utility, including safety, security, and effectiveness. “What I hoped to demonstrate in the ADLM challenge is that you can leverage some of the tools that come out of AI semantic embeddings, and merging those with traditional search functions can result in better retrieval,” Montgomery said. “AI and systems like this can … help people turn their ideas into real solutions backed up with real information.”
“We wanted to educate and build community in ADLM around data science, and I think it’s a really effective way to do that,” said Zaydman, who noted that the contests also help engage professionals across disciplines in trying to solve a shared problem.
The source code for all nine submissions is available for download from the LabDocs Unlocked GitHub page at https://github.com/myADLM/ADLM-2025-Data-Challenge. Zaydman and Bunch are planning the association’s next data science challenge, to be announced July 30 at the ADLM Data Science Symposium in Anaheim, California.
Karen Blum is a freelance medical and science writer in Owings Mills, Maryland. +Email: [email protected]