U.S. Airforce Research Lab Creates Unified Data Platform with Progress MarkLogic, Speeding Research Efforts
Products:
MarkLogicChallenge
AFRL research was hampered by a network of disparate data sources, inflexible software and lack of governance that limited research due to the lack of the ability to share knowledge and trace the pedigree of that knowledge.
Solution
AFRL deployed MarkLogic to build a unified data platform to help securely enable discovery and sharing of information.
Result
HyperThought™ eliminates many of the silos and reduces costs while easily discoverable historical tests improve efficiency across R&D.
Full Story
Challenge
The U.S. Airforce Research Lab (AFRL), the primary scientific research and development center for the Department of the Air Force, is made up of about 700 in-house researchers, supported by thousands of external researchers around the world.
Collaboration was hampered as the external researchers worked with disparate databases and tools, resulting in numerous data silos that slowed critical research and decision making. AFRL was also challenged with massive data volumes, data velocity and variety. For example, a single simulation could produce an exabyte of data in one month and the organization catalogs more than 1.5 million types of metal alloys.
The disparate data sources, inflexible software and lack of governance limited research due to the lack of the ability to share knowledge and trace the pedigree of that knowledge. What was needed was a multi-model flexible data source that would enable AFRL to add search, security, semantics and performance at scale.
We used MongoDB early on, but we realized there are so many things that you can’t do with it, like built-in search, Semantics, and true RDF expression of our data models.
Matthew Jacobsen
Systems Analyst & Developer, US Air Force Research Lab
Solution
AFRL named the project HyperThought™. AFRL was determined that HyperThoughtbe iteratively developed with agile principles, and be flexible, scalable and deployable with hybrid storage and connection models.
Early efforts with MongoDB showed that HyperThought held promise, but required a major effort to tether together critical missing pieces and derive benefit from the data. AFRL’s key priorities, search, security, semantics, performance at scale with a multi-model format were out of reach with that solution.
During the due diligence process, AFRL discovered the Progress® MarkLogic® data platform and determined it was the solution that could provide the multi-model context to cut through and unite the many disparate data sources to help speed research and promote smarter decisions faster.
Result
Combining best-of-breed commercial and open-sources technologies, HyperThought is providing AFRL researchers quicker access to data that is growing in complexity and helping them gain new insights into the data. It is a solution with two main pillars—an integration layer and systems connection layer. End users benefit from HyperThought’s extensive toolset, including content management, a digital workspace, equipment integration, workflow management and visualization. In fact, AFRL has realized a performance increase of 100X.
The MarkLogic-based platform delivers:
- Flexibility, Agility, Scalability - Adding new data sources, changing data models and increasing data volumes are now surmountable challenges with HyperThought.
- Efficiency - Historical tests and results are now discoverable and usable, improving efficiency across development and research.
- Collaboration - Scientists can now securely share data, which fosters collaboration with other researchers like Lawrence Livermore National Labs.
