GraphChi turns your PC into a machine

Computer scientists at Carnegie Mellon (USA) have developed a framework for implementing large-scale computations for tasks such as analysis of social networks and internet searches, efficiently and from a single PC.

It could help software developers dedicated to modern work on different tasks: for example, the design of a new recommendation engine using social networking connections.In order to make effective recommendations (such as “your friends liked this movie, so here’s another you have not seen it yet, but you probably want”), the software must be able to analyze the connections between the network members. This type of task is called computation graph and it is increasingly common. However, working with large sets of data (such as social networks that operate online) generally requires the processing power of many computers together, as the service offered by the Amazon EC2 cloud.


The new software, called GraphChi, takes the hard drives of large capacity which are becoming increasingly common in personal computers. A graph normally be stored in temporary memory (RAM) for analysis. With GraphChi, the hard disk performs this task.

“The PC does not have enough RAM to support a whole web graph, but have hard drives that can store a large amount of information,” said Carlos Guestrin, codirector of the Select Lab at Carnegie Mellon, where he developed GraphChi. However, when read and write data to the hard disks are slow compared with the RAM, which tends to reduce the calculation speed. Aapo Kyrölä So, a student of Guestrin, has designed a more rapid and less random access the hard drive.

According Guestrin, a Mac Mini running GraphChi can analyze the social graph of Twitter in 2010 that has 40 million users and 1.2 million connections in 59 minutes. “The previous published results on this problem indicates that took 400 minutes using a group of about 1,000 computers,” says Guestrin.

As the technology becomes more interconnected and data sets get larger, computer graphics is becoming increasingly important in many areas mcas says David A. Bader , an expert in computer graphics at Georgia Tech “Trying to understand how the human brain or try to make sense of the medical records of a patient requires computer graphics,” he says.

Graphical analysis also supports the development of new Web products, said Jeremy Kepner , a researcher at MIT (Massachusetts Institute of Technology, USA). “The search for documents, ad placement, route planning, travel reservations, cyber security and everything else depends on the graphical analysis,” he explains.“Allowing Web developers do these tests on their desktop computers acts as a catalyst for these industries and accelerate product development.”

Guestrin GraphChi adds that can handle graphics streaming , more accurately modeling large networks, showing how relationships change over time. Bader and others at Georgia Tech have created a framework for computer graphics, called Stinger, supercomputers optimized for working with large graphics streaming .

“The scale of these problems, obviously, will continue to grow,” said Guestrin. However, claims that GraphChi has capacity for handling many computer graphics problems without resorting to large-scale cloud-based solutions or supercomputers.

“A researcher in computational biology could make large-scale computations on your PC, a developer working on an algorithm of data center can test it on your laptop before uploading to the cloud,” said Guestrin. “There are large amounts of data everywhere, but some are not as great as before, in relative terms. Tools like GraphChi allow many companies and start-ups meet all your computing needs in one machine graph. is profitable and also drives innovation, “he concludes.