Archive | News

RSS feed for this section

Introducing Ogma, the Javascript library for large-scale graph visualization and interaction

Linkurious announces the release of Ogma, the Javascript library for large-scale graph visualization and interaction.

State of the art visualization libraries for the Web are unable to display graphs with more than 10 000 nodes and edges. With Ogma is it now possible to display more than 100 000 nodes and 100 000 edges. Ogma is compatible with more than 80 browsers and any devices. Companies can now start integrating a reliable and scalable graph visualization component into their applications.

Ogma performance - large-scale graph visualization

Technology

Modular architecture

Ogma makes visualization of large graphs possible by leveraging the full performances of WebGL combined with a modular architecture. The visualization engine is built for WebGL first ; it supports HTML5 Canvas and SVG with the same level of details, though performance are dramatically limited by these rendering technologies.

Dynamic graphs

The library is designed for graphs with living data where nodes, edges, and their properties can change in real time due to user interactions or updates from data sources.

Level-of-details

A key challenge in visualizing large dynamic graphs is to display as little information possible on screen to avoid getting overwhelmed by the amount of data.
The graphics engine of Ogma provides the ability to define visibility thresholds for various visual components. For example, node texts can be displayed if the node is big enough on screen.

Ready to code with

Developers can get started easily using a unified API and an event model carefully designed. They get access to an extensive documentation website that contains the documentation reference of the API, more than a hundred examples, and advanced demos. They usually get answers in less than a single business day through the support channel.

Ogma API documentation

Ogma API documentation

Developers get access to the Ogma Showcase. It is a complete application written in Angular 2 to demonstrate the main features of Ogma. They can load sample graphs into the Ogma Showcase, play with layouts, filters, and styles, and export Javascript code snippets to quickly get results.

Ogma Showcase screenshot

Ogma Showcase screenshot

Integration

Ogma can be integrated into any modern Web-based application. It supports module bundlers such as WebPack, it can be used with React, and provides Typescript definitions for fast integration with Angular 2.

Ogma and its layout algorithms can be run on a server within a Node.js application. It provides a solution for clients running on slow machines so they can delegate computationnally expensive operations to the server.

Features

Ogma comes with a complete set of features, including:

Load your data

  • Call the graph API to add nodes and edges to the visualization.
  • Import data from multiple file formats.
  • Connect to a Neo4j server using the Bolt protocol.
  • Or generate synthetic graphs using Wattz-Strogatz or Barabasi-Albert models.

Customize visual results

  • Apply graph layouts (force-directed, hierarchical, concentric, geospatial).
  • Provide built-in user interactions with the mouse, trackpad, or through the API to hover, to select, to drag, or to group nodes and edges.
  • Customize style of nodes and edges with both basic visual variables (color, size, shape, icon, image) and advanced visual components (badges, stroke, outline, halo, and pulse effects).
  • Customize style of texts of nodes and edges (font, color, position, style, size, background) with support for line wrapping and line breaks.

ogma-example-customization

Help users analyze graph data

  • Create data-driven styles: map properties of nodes and edges to visual variables such as colors, sizes, and icons for easy and always up-to-date customization.
  • Filter nodes and edges, with the ability to create compound filters with AND/OR operators.
  • Group nodes and edges manually or with specific rules.
  • Find shortest paths.
  • Display a legend to communicate easily the meaning of the visual variables.
  • Display tooltips on user interaction to provide contextual information on nodes and edges, or to implement a context menu.

Export your results

  • Export customizable image files to SVG, PNG, JPG, TIFF which can integrate a custom watermark.
  • Export data in multiple file formats.

Roadmap

We have focused our efforts on building the next generation of interactive graph visualization engine for Web-based applications. We are now improving the scalability and quality of existing layouts. We will provide you more information during the coming months.

Deprecation of linkurious.js

Ogma comes with all the features of linkurious.js, which becomes deprecated. We will continue to support existing customers of linkurious.js until they migrate to Ogma.

How to get it

Ogma is available in a proprietary license only. Contact us to evaluate it!

Integrating graph visualization in Oracle BI with Linkurious

Our partner Peak Indicators explains how to integrate graph visualization powered by Linkurious in Oracle Business Intelligence (Oracle BI).

Linkurious + Oracle BI

Oracle BI provides powerful, visually appealing analytics. It helps people explore new insights and make faster, more informed business decisions. Like other business intelligence, Oracle BI is particularly suited for the analysis of tabular data via histograms, bar charts, pie charts and other such analytical visualisations:

oracle bi dashboard

An Oracle BI dashboard.

Peak Indicators has worked on extending Oracle BI by integrating it with Linkurious to deliver new embedded graph visualisations.

Why Graph Visualisation?

Whilst BI tools are typically designed for analytical reporting (trends, patterns, comparisons etc), graph tools are specially designed to visualise the connections and relationships that exist within your data (the visualisations are made up of “nodes” and “edges”).

The combination of BI and graph technologies offers a great way to view any aggregated data and the underlying connections it contains at the same time, with the additional capability to explore and discover all the links between the various data entities.

The combination of both helps generate new insights as we will see in an example.

Example: analysis of compliance issues

Imagine a financial services company which is analysing training feedback for some of its mandatory training. Good training results mean better compliance and less risks for the company.

Analysing training courses evaluation and compliance issues.

Analysing training course evaluation and compliance issues.

In the above scenario, Oracle BI is presenting the user with summarised trends of information with conditional formatting to highlight certain data exceptions. Here we can see an overview of how the training programs are performing.  There are 3 issues being highlighted:

  • Tony Cameron, a trainer, has received poor ratings
  • The Compliance training course has received poor ratings
  • There was a surge of compliance issues in March 2016

The first two issues are related to training delivery, whereas the compliance issues are related to business performance.   Are there any relationships between these seemingly separate issues that Oracle BI has highlighted?    Can we actually prove that poor training delivery can result in poor performance within the business?

This is where the Linkurious graph visualisations come in to play since we need to understand how all the various data entities are related to each other.   For example, in this instance the graph visualisation is showing the connections between:

  • the trainer (green)
  • the training delivery (purple)
  • the training attendees (brown)
  • the attendee’s resulting compliance issues (blue)
The connections between Cameron his students and the compliance issues.

The connections between Cameron his students and the compliance issues.

So we can see that the trainer Tony Cameron, who received poor feedback ratings, is related to 8 compliance issues within the business.

Here is a screenshot of how it actually looks when embedded within Oracle BI:

Linkurious embedded in Oracle BI adds graph insights.

Linkurious embedded in Oracle BI adds graph insights.

The analysis gets even more interesting when we add another trainer, Geoff Fraser, on to the visualisation.  Geoff has received much better ratings and we can see immediately that he is related to only 1 compliance issue in the business whereas Tony Cameron is related to 8!

linkurious oracle bi integration detail

So by using Linkurious we can deduce that improving the quality of our trainers could have a very real and positive impact on reducing the number of compliance issues!

Peak Indicators was able to quickly integrate graph visualization capabilities into Oracle Power BI using the Linkurious REST API,

Linkurious makes it easy to understand complex connections, allowing businesses to make smarter decisions. With its REST API you can add new capabilities to existing solutions like Oracle BI. Want to learn more? Contact us.

Manage software dependencies with graph visualization

Discover how Pathfinder helps developers analyse software dependencies using the linkurious.js open-source JS graph visualization library. In this article Michele Sacchetti, Head of Configuration Management and DevOps team at Facilitylive, explains why he built Pathfinder and chose linkurious.js.

Introducing Pathfinder

Almost every software program has third parties dependencies. As a software project grows with new features, software dependencies grow as well. And with dependencies, the project becomes more complex to manage. Some companies like Facilitylive manage multiple software projects. Here, the code may be split in several modules across different teams, which are sometimes geographically distributed. This makes “legacy” software dependencies and their size an even more serious problem.

For these reasons, developers introducing new software dependencies should always check if they will conflict with existing ones or if a former version of the same library is already in use. Change Request over core legacy code must be deeply analysed to know all the pieces of code impacted by such change. Some teams adopt lifecycle tools like Maven to provide partial control on dependency management, but they lack the overall view on the whole project ecosystem.

Pathfinder helps developers and release managers (or any other role in a DevOps-oriented team) address these kinds of issues in a quick and intuitive way.

Pathfinder integrates in a company Continuous Integration (CI) chain and collects all dependency information at each build, aggregating them in a global view.

The graph of the current develop branch of the Pathfinder project, you can notice the different node colors, each one representing a different dependency of the project.

The graph of the current develop branch of the Pathfinder project. Notice the different node colors, each one representing a different dependency of the project.

Pathfinder is organised into three elements for a flexible integration over different CI architecture:

  • A Client, actually in the form of a Maven plugin, collects build data and send it forward to the central server
  • A Server, which gather all informations from several clients/builds and aggregates them. Due to its relations-oriented data structure data is stored into an embedded Neo4J server. It provides the performances needed to store an high number of nodes/relations. Communication flow between client-server and server-web interface is exposed via a JSON REST API. All basic actions to manipulate or show graph details can be managed using simple http calls.
  • A User interface, in the form of a standalone JavaScript application, which provide end users with an intuitive relational graph with all dependencies information. All graph functionalities are rendered via the linkurious.js graph visualization library which allows a quick and flexible data manipulation.

Clients and server usually work in the background, connected to the CI chain and collecting information and expanding dependency analysis coverage. The user interface provides end-users with tools to analyse the data.

Graph visualization makes dependencies easier to understand

Many developers work on a few projects at once and are not aware of all other teams’ work. To have a full picture of all software dependencies, they should analyze all projects’ candidates which have direct or indirect relations with their work and note down each dependency which may cause conflicts. It is not trivial for a single project but it may become almost impossible to accomplish, and even harder to maintain, over all available dependencies.

Integrating Pathfinder analysis in the CI chain ensure all projects dependencies are gathered and sent to the server to be processed and updated, while the graph visualization interface provides an immediate feedback on many-to-many relations across all possible dependencies.

Moreover agile developers may create additional development branches over the code to isolate their works or to experiment new ideas. This approach gives birth to “branch-packages” containing such experimental code which should be tracked carefully since not production-ready.

In one glance Pathfinder provides useful informations about your project’s dependencies. Nodes color allow users to distinguish types of software dependencies and their status. They can focus their attention on areas which are under development or comes from separate development branches:

  • RED artifacts are the SNAPSHOT packages, the ones which need the most care, if you are a developer or a configuration manager. Keeping them under control and being sure your release package doesn’t contain any of them is crucial.
  • YELLOW artifacts are the BRANCH packages, which means any stable artifact which doesn’t respect the standard regular expression of Maven versions. They still need some care to make sure they are stable code and not some Release Candidate or even worst, feature branches.
  • BLUE artifact are well-know standard release code, you can usually trust them.

The Pathfinder interface built with linkurious.js makes it easier to accomplish tasks like graph filtering, impact analysis or dependency analysis.

Graph filtering

Large software projects may contain hundred of dependencies making analysis harder. That’s why Pathfinder provides a filtering interface which allow users to filter nodes based on their Maven attributes.

Pathfinder project analysis, but with a filter which hides all dependencies of inner nodes.

Analysis of the Pathfinder project with a filter which hides all the core node’s dependencies.

Both inner and outer nodes can be filtered separately to achieve the best granularity for your analysis.

Pathfinder project analysis, this time excluding all third party dependencies to show just it’s own artifacts.

Pathfinder project analysis, this time excluding all third party dependencies to show only Pathfinder’s own artifacts.

Impact analysis

If your software ecosystem is large enough, you will struggle to evaluate the impact a code modification may have over other teams’ code. Changing a method behavior, even in an apparently insignificant way, can wreak havoc on your company if you are not 100% sure about which other project are invoking such method and why. Just imagine adding a couple of log lines to increase execution feedback, just to discover (usually too late) that your method is used in several time-critical applications where the few milliseconds of delay introduced by printing logs are causing deadlines to be missed. Pathfinder impact analysis helps you answer such question: given the Maven attribute of the artifact you are modifying, it will output the graph of all other artifacts which have a relation with it within a customizable number of “hops”.

Impact analysis showing all nodes with a relation with the “pathfinder-rest-client” artifact.

Impact analysis showing all nodes with a relation with the “pathfinder-rest-client” artifact.

Software dependencies analysis

In medium/large project different developers will likely re-use the same libraries for similar purposes in different projects. But, since they will probably do so at different times, you can be pretty sure they will include different versions of the same artifact.

Pathfinder can analyze the whole dependency graph of your projects and provide you with a detailed report of duplicated inclusions with version mismatch. The same report can also be converted into Maven dependency management syntax for better centralized inheritance.

Dependencies analysis highlighting some duplicated dependencies with different versions.

Dependencies analysis highlighting some duplicated dependencies with different versions.

Pathfinder and linkurious.js

Pathfinder adopted the linkurious.js JavaScript graph visualization library as its UI rendering engine from the very first release. As stated above, Neo4j was a natural choice as a backend service to store the data model but even if it also provided an amazing graph browser interface, it was immediately clear that a more fine tuned and customisable graph rendering engine was mandatory to create a rich user experience.

The first version of the Pathfinder graph was directly connected to the Neo4j REST API via native linkurious.js methods. This gave me the possibility to concentrate immediately on the look and feel and usability I had in mind with no additional effort in writing dedicated method.

Then, once the project got its own API implemented, it was really easy to connect them to the already existing UI simply shifting to custom jQuery Ajax call.

Being more of a backend java developer, linkurious.js provided me with the perfect balance of abstraction and customisation flexibility to implement the pathfinder UI in a quick and effective way, keeping also an eye on rendering performances.

Pathfinder is one of the open-source and commercial projects which integrates linkurious.js. Are you trying to help your users understand graph data? Use linkurious.js to quickly build a custom interactive graph visualization interface.

Linkurious brings DataStax Enterprise Graph technology “to life”

Linkurious now fully supports DataStax Enterprise Graph. We’re very excited to partner with DataStax to help their clients find value in their data and enable them to use it for fraud detection, cyber security and intelligence analysis.

DataStax logo. (PRNewsFoto/DataStax)LINKURIOUS_LogoRVB_small

DataStax Enterprise is a graph database solution based on the open source Apache Cassandra architecture. Datastax provides support for key-value, tabular, JSON/Document, and graph data models. They are one of the leading graph database solution worldwide and offer many interesting features and advantages in terms of scalability and reliability for cloud applications. They already have many big clients, and cover a vast array of different use cases. For example, among others, they help Netflix deliver personalized content continuously to their 50 million users and Microsoft power its Office 360 store. More information on DSE Graph here.

Linkurious is a graph visualization solution that helps analysts around the world search, explore and investigate graph data. Supporting the new DataStax Enterprise Graph helps bring Linkurious’ analytic powers to investigate massive datasets of billions of objects, spanning across hundreds of machines in multiple datacenters. Linkurious Enterprise is easy to setup and provides advanced security and customization options. It is used for fraud detection, cyber security, intelligence analysis, medical research or any other field involving large amounts of connected data.

As Billy Bosworth, CEO of DataStax, explained right after the 2016 DataStax Europe Summit’s keynote: “Linkurious really brings the graph technology to life, and that inspired a lot of people.” 

 

2b0c61d“The use of powerful and reliable graph databases solutions like DataStax will continue to grow as data collection continues to increase exponentially in many industries. At Linkurious, our aim is to help companies make smarter decisions by revealing the value in their graph data and making it accessible and comprehensible, even to non-technical users.” Sebastien Heymann, CEO of Linkurious

 

Here is Billy Bosworth (CEO of Datastax), Matthias Broecheler (Director of Engineering for DSE Graph) and Jean Villedieu (co-founder of Linkurious) discussing how Linkurious and DataStax are working together to make graph data technology come to life at the DataStax Summit Europe 2016 Keynote:

Want to explore and understand your graph data? Simply try the demo of Linkurious or get in touch !

Linkurious sponsors GraphConnect Europe 2016

Linkurious is glad to sponsor Neo Technology’s GraphConnect Europe 2016, taking place on April 26th in London. Join us to discover how graph database technologies are changing the world and how Linkurious can help you understand the connections in your data. You’ll be able to see Linkurious in action and particularly how it can be used for intelligence analysis, fraud and money laundering investigation and cybersecurity.

You will also get to learn how Neo4j and Linkurious played a central role in the Panama Papers investigation process led by the International Consortium for Investigative Journalism (ICIJ) with Mar Cabra, their head of Data & Research Unit.

GraphConnect

Now in its third year, GraphConnect is the only conference that focuses exclusively on the rapidly growing world of graph databases. GraphConnect attracts a vast array of speakers from the developer community as well as leading organizations around the world.
This year’s edition will offer three parallel tracks to better serve developers, architects, IT executives and database and project managers: Business Impact of Graphs, Neo4j Deep-Dive, Neo4j in Action. In the last track, speakers will describe how each of them has successfully used graph databases to solve a previously intractable problem in their domain of expertise :

  • Enterprise Data Management with a Flexible Graph Platform, Axel Morgner, Structr
  • How Go and Neo4j enabled the FT to Deliver at Speed, Dan Murphy, Financial Times
  • An Integrative Framework for Data Management in Systems Biology and Medicine, Irina Balaur, European Institute for Systems Biology and Medicine

Our team will be here to meet with the Neo4j community and talk about graph visualization. Come say hello!

Panama Papers: How Linkurious enables ICIJ to investigate the massive Mossack Fonseca leaks

Linkurious is a partner of the International Investigative Journalist Consortium (ICIJ) since the Swiss Leaks scandal. ICIJ network of 370 journalists is using Linkurious to investigate the Panama Papers. Learn the inside story of the biggest data leak investigation in history.

Turning leaked documents into exploitable data

End 2014, an anonymous source contacted a Suddeutsche Zeitung Journalist, offering to leak the data of a shady legal firm based in Panama called Mossack Fonseca. More than 11.5 million documents were sent encryptedly to the Suddeutsche Zeitung, representing around 2.6 terabytes of data, documenting 214 488 offshore structures that were created and administered by Mossack Fonseca between 1977 and 2015. Internal and external communication, all of the firm’s databases, scanned documents: nearly all the documents ever created and collected by Mossack Fonseca over the years were leaked, according to Gerard Ryle.

leak-structure
Source: Le Monde

Suddeutsche Zeitung quickly realised it could not handle the exploitation of such massive amounts of data on its own and decided to get ICIJ on board. ICIJ had already demonstrated its ability to coordinate worldwide investigations numerous times in the past with the Offshore Leaks, the Lux Leaks and more recently the Swiss Leaks, for which they deployed Linkurious Enterprise. ICIJ mobilised “around 370 independent journalists from more than 100 media partners in over 80 countries”.

The initial challenge for ICIJ and the Suddeutsche Zeitung was to make these massive amounts of data exploitable by their network of investigative journalists around the world. This implied turning 11.5 unstructured million documents into something meaningful for journalist investigators.

Through its Data & Research unit, ICIJ provided the data analysis expertise to make the documents exploitable by reporters. They extracted the metadata of documents using Apache Solr and Tika, then connected all the information together using the leaked databases, creating a graph of nodes and edges. The data was stored in the Neo4j graph database edited by our partner Neo Technology. The result provides unique insights into the offshore banking world showing the relationships between banks, clients, offshore companies and their lawyers. This graph includes the numerous public personalities worldwide who have used Mossack Fonseca’s services such as Sigmundur Davíð Gunnlaugsson, the current Prime Minister of Iceland, or Vladimir Putin’s close associates among others.


How Gunnlaugsson hides his secret assets.

Shedding light on the offshore world

Once the database was created, ICIJ could start making it available to the different investigative teams globally. But their last challenge was to make the data exploration accessible to non tech-savvy investigators in a easy, secure and collaborative way. This is where Linkurious came in. Linkurious provided a simple and powerful interface that hundreds of investigative journalists around the world are using to carry out their investigation.

Linkurious Enterprise was the best solution to make the data available to our global network of more than 370 journalists. It enabled them to uncover persons of interest and potential stories in a short timeframe. Using the collaboration features and intuitive interface of Linkurious, they were able to easily expose many complex offshore structures and financial crimes“, Mar Cabra, Data Editor at ICIJ explains.

Using Linkurious, journalists were able to easily explore the connections of the entities they were investigating and expose the complex financial structures mounted by Mossack Fonseca’s clients to cover up their financial crimes. The interface of Linkurious Enterprise also allowed ICIJ’s journalists to share visualizations and collaborate throughout the investigation. Finally Linkurious Enterprise provided security features particularly important for journalists working in countries where reporting on a political scandal could have dire consequences.

We wanted to protect the identity of our journalists and make sure our data would not end up in the wrong hands. This was particularly difficult as we are exposing political figures in countries where the freedom of press is all but guaranteed. Linkurious Enterprise allowed us to work securely while working in around 80 countries

Cabra comments.


The network of middlemen and companies hiding Putin’s wealth

The methodology used by the journalists is quite simple, they compiled lists of high profile individuals (politicians, criminals, VIPs) and their first circle. They then tried to identify links between these individuals and offshore entities. Linkurious enabled them to connect the dots together and rapidly do fact checking.

When the time came to publish the story, ICIJ used the Linkurious Enterprise API to create and embed graph visualizations into its article. This way readers around the world could quickly grasp the complexities of the financial networks unmasked by ICIJ.

Using Linkurious with the Neo4j graph database is a powerful combination to detect and investigate fraud or money laundering schemes. Linkurious helps investigators easily identify and analyze suspicious connections to fight fraud. The whole Panama Papers data will be released in early May with Linkurious. Follow the stories!

About Linkurious

Linkurious helps organizations use their data to uncover relationships among people, places, things, and entities. It provides new capabilities to extract information from large and complex datasets. Linkurious helps organizations like NASA, the French Ministry of Finances or various Fortune 500 companies in fraud detection, network management, cyber-security or medical research.

Today, Linkurious Enterprise makes graph visualization accessible to all organizations through an easy-to-use interface. Compatible with Neo4j and other graph databases, Linkurious Enterprise is an on-premise platform that allows users to interact with their data to investigate patterns, discover hidden insights and reveal opportunities through visual querying. Organizations can quickly unlock the value of their graph data with a platform that includes security and collaboration features.