Yale University Shares Open-Source Code for LUX Cross-Collection Discovery Platform

Until recently, it was impossible to perform a single search across all the collections held by Yale University’s library and museums. That all changed in 2023 when Yale University publicly launched LUX: Yale Collections Discovery. LUX is a first-of-its-kind cross-collection discovery system that uses a linked data model to enable search and discovery across disparate collecting units. Now, Yale University has shared the open-source GitHub repository for its LUX platform, licensed under Apache 2.0. Yale hopes that sharing the project will enable other institutions with diverse cultural heritage collections to quickly adopt the innovative solution to facilitate cross-collection discovery.

 A screenshot of the LUX user interface

Yale’s LUX Project Origins

The Yale University Library dates back to the founding of Yale College in 1701. Over the centuries, the library has grown significantly, acquiring vast collections and expanding its facilities.

The library boasts a collection of more than 15 million print and electronic resources, including books, journals, manuscripts, maps, photographs and more. Yale University Library's collections cover a wide range of subjects and disciplines, supporting research and learning across the humanities, social sciences, sciences and professional schools. The library system comprises multiple libraries spread across the Yale University campus in New Haven, Connecticut.

However, Yale’s collections reach well beyond the library system:

• The Yale University Art Gallery was founded in 1832 when artist John Trumbull donated over 100 of his paintings to Yale. The gallery's collection has since grown to over 200,000 objects, spanning ancient to contemporary art from around the world. Celebrated for its encyclopedic collection, which includes works by artists such as Van Gogh, Picasso and Pollock, the museum serves as a vital resource for scholarship, teaching and public engagement.
• The Yale Peabody Museum of Natural History was founded in 1866 through a gift from philanthropist George Peabody. It houses an extensive collection of over 13 million specimens, spanning disciplines such as anthropology, paleontology, zoology and mineralogy. Renowned for its scientific research and public education initiatives, the museum continues to be a hub for exploration and discovery in the natural sciences.
• The Yale Center for British Art was founded by Paul Mellon and opened to the public in 1977. The center houses one of the most comprehensive collections of British art outside the United Kingdom. Its collection spans from the Elizabethan period to the present day and includes paintings, sculpture, prints, drawings, rare books and manuscripts. The center serves as a hub for scholarly research, exhibitions and educational programs, fostering a deeper understanding of British artistic heritage.

Yale’s cultural heritage collaboration efforts began over a decade ago, including announcing its Open Access Policy and launching initial cross-collections discovery efforts in 2011. In 2016, Yale appointed Susan Gibbons as its first Vice Provost for Collections and Scholarly Communications, with a focus on fostering connections within Yale’s collecting units. In 2017, a new level of collaboration between central IT and the collecting organizations began with the establishment of the Cultural Heritage pillar. The cross-functional collaboration of this team soon led to a centrally supported project to improve digital access to Yale’s collections.

Designing LUX for Effortless Public Access

In 2018 Yale had now begun its 5-year effort to transform cross-collection discovery. A central challenge was that each collecting unit (library, museum, gallery, archive, etc.) has its own collections management system. Descriptive practices vary between museum, archive and library collections. They can also differ across domains. For this reason, each unit has its own purpose-built search application tailored to their audience. Therefore, to search across Yale’s collections for a given interest area, a user might have to search four or more different websites. The team envisioned a rich user experience where users could explore diverse collections based on shared concepts and relationships. It became clear to the team at Yale that standard search technology would not be enough. Linked Data—and a Graph Database to support it—would be a key enabler to drive the user experience. To help drive the Linked Data strategy, Yale hired Rob Sanderson as Senior Director of Cultural Heritage in 2020.

As the project moved into architecture design, the technical requirements now included a document database, search engine and graph database. Yale explored two pathways: the multi-system integration approach and the multi-model database approach. In the multi-system approach, Yale would be responsible for selecting multiple technologies and performing the systems integration needed for the technologies to perform according to the LUX use case. In a multi-model database approach, Yale would choose a technology that natively supports text, document and graph with a single backend. In 2021, following a proof-of-concept phase featuring several search engines and graph databases, Yale chose the Progress MarkLogic Enterprise Data Platform due to its native multi-model database and search capabilities. With this approach, Yale saw an opportunity to focus more time on frontend development and less time on backend integration tasks.

The MarkLogic platform now serves as the Yale LUX backend, enabling advanced data exploration in a knowledge graph model. For more detail on the origins of CHIT and Cross-Collection Discovery at Yale, watch this presentation from Larry Gall of the Yale Peabody Museum.

LUX Go-Live: Bringing Centuries of Art and History to the Public

The LUX Project went live to the public in June of 2023. Yale’s LUX website describes the platform as follows:

“LUX is a digital platform that enables users to search across Yale’s museums, archives, and library collections. This includes works of art, archives, scientific specimens, and other cultural heritage items held at the University. Since its foundation in the early 18th century, the University has encouraged, accumulated, and actively built its collections in the service of its educational mission. LUX supports this mission by making many of these objects and resources accessible so that new knowledge and understanding can be built by those within and beyond Yale.”

Rob Sanderson, Yale’s Senior Director For Cultural Heritage, had the following to share with Yale Daily News: “The power of LUX comes from its ability to allow users to uncover hidden relationships between objects, from shared concepts to famous figures in Yale’s collections…this groundbreaking tool brings together all the university’s collections through modern technology that will enrich discovery opportunities for users, making it easier for them to find what they’re looking for and explore new objects.”

“We think LUX is game-changing for the cultural heritage domain,” he added. “No other institution has a system with this level of capability.”

Now, Yale is sharing the code of its LUX discovery platform to give more cultural guardians a blueprint for opening their collections to the public. This way, more of the treasure trove that is the world’s finest art and body of literature can be experienced at our fingertips. Imagine, you could enter the courts of the wise man, commune with the brightest minds and tap into the wealth of generational knowledge carefully collected and preserved over the years.

To learn more, you can explore LUX for yourself and view our webinar with LUX stakeholders.

For questions, reach out to James Cahill at james.cahill@progress.com