Frequently Asked Questions

DataWalk is an Enterprise-class software platform for rapidly connecting numerous large data sets into a unified view/repository, for fast data access, visual analysis and investigations.

In general DataWalk is for teams:

• Doing intelligence analysis, fraud detection/investigation, AML alerting, and KYC
• With multiple power user analysts
• Who have a need to do analysis of data that spans many systems/sources
• That value a collaborative, Enterprise approach for the analysis system

Our customers include U.S. Federal Government agencies such as DoD, DHS, and Department of State; leading financial institutions (e.g., a top-20 bank in the U.S.); national intelligence agencies; insurance companies, larger municipal law enforcement agencies, and various other commercial businesses.

In general DataWalk enables you to get much better results, much faster. Specifics depend on your particular application and use case.

In many environments data is spread across multiple silos, such that it is a slow, manual process to locate, collate, and query/analyze the desired data. In DataWalk all data is pre-connected and at your fingertips for access, which in itself saves a significant amount of time. DataWalk then further accelerates results by automating key analytical processes; enabling users to easily generate, understand, and quickly run complex queries; and to execute graph algorithms that quickly automate the process of finding distant connections and automatically finding patterns indicative of organized crime groups across all your data.

No!! DataWalk has powerful visualization capabilities, but DataWalk is built to enable you to generate analytical products. Our system is uniquely suited for analysis (e.g., advanced visual querying of complex data, easily generating and modifying sophisticated scores, easily integrating scripts for complementary functionality via the App Center, and graph analytics).

No! DataWalk includes a robust link analysis facility, but link analysis is only one of DataWalk’s capabilities. Others include rapid data integration; easy creation (and quick execution) of complex visual queries; graph analytics across vast amounts of data, an end-to-end solution for machine learning using real data, and serving as an intelligence database.

DataWalk enables you to build analytical products on top of the system. You can insert other applications into DataWalk, and you can insert DataWalk in other applications.

DataWalk can be part of a real-time decision making process via API integration.

Yes! DataWalk includes an embedded big data repository where any or all of your data can be connected, stored, and securely accessed for instant 360-degree views leveraging all of your data. One place to look for any of your data!

Yes, this is enabled by DataWalk Quick Search, which is available for both desktop and mobile devices. This of course requires that users have appropriate permissions, and all such accesses are logged.

Yes. DataWalk supports an NLP facility for entity extraction; the ability to process data in foreign languages, and the ability to automatically identify patterns in text content. DataWalk also can effectively parse documents in PDF, Word, and other formats.

Yes. DataWalk has existing integrations with many such services, and integrations with new services can often be generated by DataWalk in a few hours.

• DataWalk is a server-based system intended to support multiple users and use cases.
• DataWalk can support separate application environments for production and development/test.
• DataWalk can support complex integration of both technology and business processes, and fit into Enterprise environments and workflows.
• DataWalk supports an optional high availability ("HA") configuration
• Enterprise environments may have more data sources and they may be more complex.
• Enterprise systems often need to be able to scale to larger numbers of users and vast amounts of storage capacity.

Functionally, Palantir Gotham is probably the most similar competitive product (though DataWalk is far more cost effective than Gotham).

Harris/IBM i2 Analyst’s Notebook is a desktop-based system specifically for link analysis. DataWalk is an Enterprise-class system for link analysis and much more, including the ability to effectively analyze vast amounts of data. It’s like comparing a Ferrari to a motorcycle.

No. DataWalk is a server-based system accessed via a web browser.

Yes. DataWalk enables users to easily prepare data for Machine Learning, including features, vectors and training sets. Once data is prepared, it can be used as an input to customer’s models.

DataWalk includes some rudimentary case management capabilities, but most customers have existing case management systems, and our strategy is to integrate with those.

Most analysts can productively use DataWalk after taking our half-day eLearning course. Multi-day training is typically appropriate for power users.

No. DataWalk is robust Enterprise-class software, such that a downloadable self-driven trial is not practical.

This is a possibility that your DataWalk account manager can consider.

Though DataWalk is not optimized for simple drawing, it is possible to create drawings in the system.

DataWalk is an enterprise-class software platform for graph and advanced analytics. DataWalk enables you to fully understand your data and produce better data science products by rapidly connecting numerous large data sets into a unified knowledge graph for fast data access, analysis and investigation.

DataWalk can be thought of as a feature store that enables you to easily share, understand and reuse features. Features can be both tabular in nature and/or graph in nature, which is a particularly powerful DataWalk capability. Features can be created from data at scale. You can use ML on vast amounts of data stored in DataWalk without moving data.

DataWalk typically fits best for organizations that want to use context (i.e., insight about relationships) in their advanced analytics and AI projects, and who value our solution for other analytic and data management facilities (e.g., complex no-code querying, graph algorithms, knowledge graph and data store).

DataWalk covers and automates the end-to-end ML pipeline, starting from data ingestion and preparation, to feature generation, model training, deployment and monitoring.

Python

Yes! DataWalk provides an embedded and integrated Jupyter Notebook interface and offers a Python Machine Learning library that enables data exploration, visualization and machine learning with or without data movement.

The DataWalk ML facility provides out-of-the-box embedded ML algorithms that enable ML processing at scale and not embedded libraries such as sklearn and Pandas. If desired it can be expanded with any other Python packages.

DataWalk provides a variety of machine learning algorithms for regression, classification clustering, anomaly detection, time series and dimensionality reduction sorts of tasks.

DataWalk ML enables structured embedded and non-embedded ML analysis. For unstructured tasks such as text or image analysis, standard DataWalk features, 3rd-party tools and other non-embedded Python packages can be used.

Yes, all machine learning enterprise processes, such as data governance, security, performance management, and data feed workflow, are enabled and ensured by DataWalk.

Yes, DataWalk automates the machine learning pipeline at various stages:

• DataWalk ensures data consistency and no manual actions are required when new data is introduced to the system.
• When the best performing model is selected, it can be quickly deployed in production by converting it into an appcenter app.
• DataWalk ML/AI feedback is instantaneous and based on real data. When your models are deployed, then feedback is automatically captured.
• Data access control is provided continuously
• Models can be re-trained automatically with new data and then deployed in production

Yes. DataWalk enables you to access any data source with a defined interface. Many connectors are currently available, and new connectors can typically be configured in a couple hours by DataWalk field engineers. Then you can quickly link all data in a business representation through the knowledge graph capability. In addition, DataWalk embeds AI technology to automatically complete, clean, overwrite, fix and enrich data so that it can quickly be used for further analysis.

With DataWalk you do not need to think about maintaining dozens of ETL tasks created from hundreds of lines of code. DataWalk is designed to support an entire enterprise data processing pipeline and automating it in the most optimal way, such that you can eliminate the need for further optimization regardless of whether it’s feature generation or any other data manipulation.

Embedded models created in DataWalk are deployed directly inside DataWalk DB, while models not embedded are deployed using the DataWalk App Center as a container.

DataWalk accelerates Machine Learning and delivers better results by:

• Enabling ML for vast amounts of data by eliminating data movement
• Deriving new insights from relationships (context enabled by graph)
• Data governance, analytics and AI unification
• Automated data engineering pipeline
• Collaboration at the enterprise level
• Instant feedback loop

Graph Machine Learning is a set of tools used for processing network data and leveraging the power of the relationships between entities that can be used for predictive, modeling, and analytics tasks. Graph ML is a key DataWalk capability.

DataWalk is an Enterprise-class software platform built around DataWalk’s Graph/Relational Database Hybrid technology.

Yes. DataWalk can integrate with any SSO that supports SAML 2.0, LDAP, Kerberos, or Kerberos+CAC.

Yes, the DataWalk Platform is a multilayered architecture with a RESTful Interface layer. There are also other interfaces to integrate DataWalk in a larger enterprise workflow.

DataWalk is proven in highly secure environments. DataWalk has Authority To Operate (ATO) in institutions with a Top Secret data security level, as well in Federal Networks of DoD, DOJ, and others.

Security requirements vary by organization, and the DataWalk team will strive to ensure that our solution will meet your security requirements.

Yes, DataWalk collects every possible action that was performed by users. System administrators have the ability to configure information collected from 40+ different metrics.

Yes, DataWalk can be configured in a high availability configuration with no single point of failure. In addition, data backups can be performed without interrupting system operations.

Linux: Red Hat 8.5-8.9, or Amazon Linux 2

A minimum DataWalk implementation typically consists of four servers: an Application Server, an Integration Server, a Compute Server, and a Management Server. As DataWalk is a scale-out system, additional capacity can be added by adding additional Compute Nodes. The number of servers required, and the CPU/RAM requirements of each, will depend on the volume of data, number of users, and any requirements for high availability.

Storage options include cloud storage (AWS, Azure Blob), as well as specified on-premise S3-compatible storage arrays.

Yes, DataWalk can run on VMware and most other virtual environments.

Yes. DataWalk can run on AWS, Azure, and other cloud environments and supports dynamically scaling the environment by adding or deleting nodes as needed.

For connectivity between Application Servers and Compute Nodes, GbE is sufficient. For connectivity between DataWalk Compute Nodes, 10GbE is recommended.

No, though if needed DataWalk partners may be able to provide a DataWalk SaaS solution. Contact us for more details.

Yes. DataWalk connects to many sources, and if the source has a defined interface, any new data source can typically be integrated in a couple hours.

Yes, though the DataWalk backup facility should be used, with the output then collected by your backup software.

No. If needed, data can efficiently be transformed and normalized in DataWalk.

No. DataWalk uses internal technology for creation of containers, but DataWalk itself cannot yet be deployed into a container. This item is on the DataWalk product roadmap.

Yes.

No.

Yes.

DataWalk is a Commercial Off The Shelf (COTS) software platform built on proprietary DataWalk graph vectorization technology. The DataWalk platform enables you to leverage DataWalk’s unique way of computing to build internal and external data products.

No! DataWalk certainly provides the ability to visualize link charts, but this is a very small part of what we do. DataWalk’s core capabilities are around fusing and analyzing data across many data sources for fast visual querying, graph analysis, and machine learning.

DataWalk:

• Enables graph-based data exploration on multi-TB data sets.
• Can effectively and economically analyze huge graphs.
• Is uniquely suited to calculating complex clusters and automatically discovering hidden/distant relationships.
• Enables you to easily generate, understand, and quickly run complex queries which traverse multiple data sets.
• Typically enables you to apply changes to the ontology - and make it actionable - without modifying upstream processes.
• Can rapidly execute flow analytics on vast amounts of data.
• Ensures that any request executed via the DataWalk API is horizontally scalable and highly efficient.
• Delivers reliable results in dynamic environments where new data is being added.
• Delivers the above capabilities with a dramatic reduction in hardware requirements compared to solutions that operate using in-memory analytics.

Yes, DataWalk can economically and effectively analyze ultra-large graphs with billions of nodes/edges. This is enabled by the ability to quickly move data from disk drives to analytics, via DataWalk’s graph vectorization. To our knowledge DataWalk is the only commercially available tool that enables the file system to be used for these types of computations; this enables lower hardware cost by enabling computations on data that doesn’t fit in RAM.

Yes, DataWalk is uniquely suited to calculating complex clusters and automatically discovering hidden/distant relationships. With DataWalk you don’t need to worry about depth of the traversal, as you can easily reach data at distances greater than six degrees of separation.

Yes, DataWalk typically enables you to apply changes to the ontology, load new data, and - unlike a data lake, data warehouse or other mechanisms - makes it actionable without modifying upstream processes. This is enabled by DataWalk’s flexible ontology, which in turn is enabled by DataWalk’s graph vectorization capabilities.

DataWalk can rapidly execute flow analytics on vast amounts of data, as DataWalk’s unique approach to data indexing enables execution of graph and OLAP analytics at the same time, without data movement. This enables DataWalk to build pivot tables and aggregations on-the-fly.

Yes, DataWalk delivers reliable results in dynamic environments where new data is frequently added. DataWalk’s Consistency Assurance facility automatically, properly inserts new data into the index.

Yes. DataWalk has numerous connectors available for a variety of data sources, and importantly, we can typically configure integrations with new sources in days.

No. DataWalk has developed a hybrid graph/relational database.

• Graph databases require that you write an application (using a graph query language) in order to solve relationship-oriented analytic challenges. In contrast, DataWalk is a full-stack solution where coding is not required.
• Graph databases are excellent for OLTP applications, while DataWalk is far superior for complex analytical workloads. Pivots cannot be executed in graph databases.
• DataWalk is optimized to do batch analysis of nodes and edges with minimal (linear) complexity.
• Graph databases typically run in-memory, but this is not required with DataWalk.

Four key things:

• A graph database lacks the semantics to summarize data along some attributes, while DataWalk has many functions to do this.
• Speed of access/analysis when operating on sets of objects and links.
• Far superior for graph traversal.
• Automatic connection generation
• DataWalk has consistency checks, which graph databases lack

DataWalk has an Online Analytical Processing (OLAP) capability. While a graph database lacks the semantics to summarize data along some attributes, DataWalk has many functions to do this. For example, you may want to create a view of the number of sales reported by each sales rep for each of the last 24 months. This is impossible to do in a graph database without creating a program, while in DataWalk this is a built-in function.

In contrast to graph databases, DataWalk performs massive joins for graph traversal. With this approach, proper sorting of disk tables can make all operations to be merge joins instead of hash joins. This reduces the complexity of solutions from exponential O(N2) to linear O(N). Further, such operations use sequential disk access which is more efficient than traversal in graph databases, which in general requires random access to memory. This is analogous to using an array and linked list. Scanning the second one is slower than the first one because every step requires access to a random address in memory when the array just proceeds to the next memory cell.

A graph database is just that, a database. You must write programs to generate the links between objects and then store those links in the database. DataWalk allows you to specify an unlimited number of algorithmic links between objects, and the DataWalk system then generates these links automatically whenever objects are added or updated.

In general this is not practical. First you have the significant issue of building and maintaining an application, and note that building DataWalk has required hundreds of Engineering man-years of development and over ten patents. Second is that - as indicated above - a graph database simply cannot support the same functions as DataWalk.

DataWalk is architected specifically around the design principles associated with linear scalability. The linear scalability of DataWalk has been proven with deployments analyzing many billions of records, and our customers are using DataWalk to operate on data at levels previously unachievable.

No, you don't need to load all the data. Our technology will index the information necessary to support your business users and represent it as a high-hierarchy ontology. This usually means that only part of the data is indexed.

In order to execute sophisticated algorithms on vast amounts of data, it is required that the data is indexed. Attempting to execute such algorithms via a federated approach (reaching out to other data sources ad-hoc) is not practical with billions of records.

DataWalk has developed a hybrid graph/relational database. The DataWalk application is self-maintaining such that no Database Administrator (DBA) is required, and all maintenance activities are executed via DataWalk application interfaces. You should think of DataWalk as a single application with an embedded database, and you will not need to certify the DataWalk database.

No. DataWalk is based on a multi-layered architecture, and each layer can be distributed in a different network.