The VARYSS Open Source Platform Fuses Social Science with Data Science to Identify, Predict, and Influence Strategic Change in Real-Time

Our data scientists blend high-dimensional statistical analysis with cutting edge political and behavioral science to produce unique insights into the dynamic political processes of each organization they are studying. By tracking the ripples in an organization as changes and exogenous events introduce new forces that pull and push on the fabric of an organization, we can create detailed models of how these organizations are structured, who is yielding the real power, and how they will react to future stresses on the organization. We can visualize this information for our customers using a variety of graphical tools, written narratives, or periodic views of the same type of visualizations, with the goal of enlightening the customer to the dynamic processes that are at the core of the targeted organization and how those structures change over time.

Using Artificial Intelligence (AI) outside of a few well-studied problems requires combining Machine Learning (ML) expertise with deep domain knowledge and experience. Kingfisher has over a decade of experience creating models of political behavior and a global news database with information extracted from over 300 million stories in multiple languages. Through our work with this massive datasets we have refined standard Natural Language Processing (NLP) pipelines, and developed and operationalized novel measures of political behavior, enabling us to bring unique insight and predictive capability to hard political problems. Kingfisher’s cybersecurity mission partners developed novel methods that process 20,000 times more machine data than prior technologies and created an endpoint detection AI exceeding the best published performance by 15 times. Kingfisher seeks to apply our broad and deep capabilities to hard national security problems that have resisted traditional AI solutions. 

Kingfisher has deep expertise interfacing NLP pipelines with core algorithms in AI, including deep neural networks. We have focused on extracting actionable quantitative intelligence from open source media. We use topological clustering methods to interpret the output of stochastic generative topic models which allows us to discover complex and emergent topics using short histories. The combination of high frequency time series analysis and automated entity and concept discovery drives increased understanding of causes and effects, and increases the value to the customer. Our approach is centered on entity, relationship, and concept extraction, but our expertise extends beyond proficiency in standard NLP tasks such as topic discovery using stochastic generative models, and NLP primitives such as tagging, chunking, and shallow parsing. Our data holdings include open source news reporting which is extended to other domains such as social media and includes structured information obtained from over 300 million documents. Our large data holdings allow us to use semi-supervised and unsupervised approaches to address the lack of diverse tagged corpora.

Our anonymizing crawler collects several hundred thousand news events every day, collecting and processing news stories in 94 languages. This Multi-Source Intelligence as a Service (MSIaaS) blends a variety of Open Source Intelligence (OSINT) sources into a single comprehensive feed. We crawl our own proprietary list of content creators, such as newspaper and journals, collecting new events as they are published. We can collect your custom lists of content creators or social media feeds, while utilizing our distributed anonymizing infrastructure. Additionally, we can target our collections so that the requests come from specific geographic areas to avoid having content filtered or modified by the provider. Our infrastructure can handle several million news events a day, well in excess of the traditional open source media content of the globe.