Based on the survey results published in MeriTalk’s recent Big Data Forecast report, about one-fourth of Federal IT execs say they have launched at least one big data initiative. In five years, Federal IT execs predict they will spend 16 percent of their IT budget, or nearly $13 billion, on big data. Federal IT execs estimate their agencies can save an average of 14 percent of their overall budget from successfully leveraging big data – or nearly $500 billion across the Federal government, and 69 percent say big data will help government work smarter.

Not only does big data analytics help customers save money, it directly affects their mission effectiveness including improved decision-making ability and operational intelligence, fight crime, increased transparency and elimination of waste, fraud and abuse. Big data is no longer about dealing with large volumes of data; it is an opportunity to find insights in new and emerging types of data and content, to help agencies positively influence the mission, and to answer questions that were previously considered beyond our reach.

While it’s commonly known that data can help track criminals, the latest data science is helping to stop crime before it ever starts in cities across the country. Big data analytics is now being used widely by law enforcement officials to identify high crime areas and patterns of crime.

The Smart Policing program, implemented in 38 different American police departments, funds and empowers local, data-focused, crime prevention tactics. A key feature of the program is “hot spot policing” which analyzes geographic patterns to uncover highly likely crime locales. Analysis incorporates information from GIS mapping and predictive analytics to search for trends in criminal activity, and to determine its root causes. For example, from 2012-2013, big data analytics helped the Philadelphia Police Department identify at-risk areas for crime and send more regular patrols to those places. The result was a 39 percent reduction in home burglaries.

Big data analytics can help with Insider Threat Detection and Mitigation. Insider threats are not numerous, but they are certainly the most costly and extensive when it comes to damages. New advances in big data analytics utilize data mining and behavioral-based techniques to detect and stop insider threats. Powerful tools such as Splunk and Varonis help provide visibility into the human behavior changes, baseline computer behavior (volume, frequency, and patterns) as well as psychosocial risk factors to weed out insider threats before they happen. They help detect behavioral patterns based on the digital footprints that people leave behind in the course of their normal daily activities. Several government agencies like the FBI are now using big data analytics to identify patterns on an insider threat continuum using diagnostic analysis.

A number of troubling events over the last several years, including the Washington Navy Yard shooting, Fort Hood shooting by Major Nidal Hissan, Edward Snowden’s big security data breach, and the WikiLeaks disclosures have called into question the procedures for providing access to facilities and documents to government employees. The Army is testing a program called Automated Continuous Evaluation System. Utilizing advanced big data analytics solutions and context aware security, the system analyzes government, commercial, and social media data to uncover useful patterns. The program revealed 21.7 percent of those in the pilot had not disclosed important information like serious financial problems, domestic abuse, drug abuse, or allegations of prostitution. In about 3 percent of cases the charges were serious enough to result in the suspension or revocation of their clearances.

Hadoop is another technology that is gaining a strong foothold in the big data space. The primary usage of Hadoop is for processing massive amounts of data in a scalable manner. Hadoop enables the agencies to organize and process large amounts of data while keeping the data on the data storage cluster. Typical enterprise use cases include business analytics, extraction, transformation and loading/data warehousing, log analysis, and Web search engines.

With all the benefits that big data can bring, it’s no wonder why state, local, and Federal organizations have begun to harness it as a powerful tool not only in decision-making, but as a means to enhance security and foster a better relationship with the public. The impact of big data will be transformational; the path to effectively harnessing it does not require the agencies to start from scratch with green field investments. It is clear that the government can build iteratively on the capabilities and technologies it already has in place. Agencies looking to leverage big data solutions will have to start evolving their IT infrastructures including massively scalable storage and network infrastructure designs, as well as considerations for data protection, data sharing, data reuse, ongoing analysis, compliance, security/privacy issues, data retention and availability.

Federal IT professionals report, on average, that it will take their agency at least three years to fully take advantage of big data. Have you started your big data journey yet? Are you ready to leverage the power of big data in you agency? The time has come to start thinking about big data and how it can help your agency support mission outcomes. I would encourage you to reach out to the big data experts at Government Acquisitions and let them help you realize the full potential of your data to positively influence and further your agency mission.