Getting the most from big data is an ongoing challenge. The Defense Advanced Research Projects Agency (DARPA) last Friday selected of five participants for its Hierarchical Identify Verify Exploit (HIVE) program, announced last summer, intended to develop a new high performance data handling platform.

“Today’s hardware is ill-suited to handle such data challenges, and these challenges are only going to get harder as the amount of data continues to grow exponentially,” according to Trung Tran, a program manager in DARPA’s Microsystems Technology Office (MTO) heading up HIVE. The goal is to develop a “powerful new data-handling and computing platform specialized for analyzing and interpreting huge amounts of data with unprecedented deftness.”

Selected for the project are: Intel Corporation (Santa Clara, California), Qualcomm Intelligent Solutions (San Diego, California), Pacific Northwest National Laboratory (Richland, Washington), Georgia Tech (Atlanta, Georgia), and Northrop Grumman (Falls Church, Virginia).

“The HIVE program is an exemplary prototype for how to engage the U.S. commercial industry, leverage their design expertise, and enhance U.S. competitiveness, while also enhancing national security,” said William Chappell, director of MTO, in the release announcing the selections. “By forming a team with members in both the commercial and defense sectors, we hope to forge new R&D pathways that can deliver unprecedented levels of hardware specialization.”

As described by DARPA, a core HIVE goal is creation of a “graph analytics processor which incorporates the power of graphical representations of relationships in a network more efficiently than traditional data formats and processing techniques according to DARPA. Examples of these relationships among data elements and categories include person-to-person interactions as well as seemingly disparate links between, say, geography and changes in doctor visit trends or social media and regional strife.”

“In combination with emerging machine learning and other artificial intelligence techniques that can categorize raw data elements, and by updating the elements in the graph as new data becomes available, a powerful graph analytics processor could discern otherwise hidden causal relationships and stories among the data elements in the graph representations.

DARPA suggests such a graph analytics processor might achieve a ‘thousandfold improvement in processing efficiency’ over today’s best processors, enabling the real-time identification of strategically important relationships as they unfold in the field rather than relying on after-the-fact analyses in data centers.

Link to the DARPA release: http://www.darpa.mil/news-events/2017-06-02

Link to more information about HIVE: https://www.fbo.gov/index?s=opportunity&mode=form&id=daa4d6dbee8741f56d837c404eac726d&tab=core&_cview=1

Image source: DARPA