What do JRR Tolkien, JP Morgan Chase, the military, and rescue workers have in common? Palantir.
"The Palantír" is the title of the 11th chapter of Tolkien’s The Two Towers. The name refers to the "seeing stones" that allow one to view what is happening elsewhere. In 2004, the name was also taken on by a company that develops software organization to extract meaning from various streams of data to combat terrorism, fraud, and disaster damage.
Palantir distinguishes its approach from data mining by calling it "data surfacing." To solve what it identifies as nothing less than the "world’s hardest, most complex, critical data problems," the software has to make sense of data that comes in from assorted streams in divergent and imperfect forms. It also has to detect the trail of data that has been deliberately obscured by people who want to conceal their movements:
In these contexts, the algorithmic approach fails. So we do something else. Our data platforms are designed to surface the totality of known data about a problem in a way that's easily digestible by the best pattern matching and inference machinery ever devised: the human brain.
Due to its abilities to uncover what eludes normal pattern detection, Palantir software is used by the CIA, FBI, the New York City Police Department, and the military to track terrorist and criminal activity. It is also used by financial institutions like JP Morgan Chase as a defense against fraud. The bank has used the software to foil hacker attacks on client accounts and ATMs and awarded its Hall of Innovation Award to Palantir in 2010.
The Google of big data In 2011, JP Morgan Chase’s CIO, Guy Chiarello, observed that Palantir make "gold mines" out of "data landfills." He compared it to Google, saying that Palantir reveals what is "inside the walls of corporate data" in the same manner as the search engine uncovers what is on the Internet. The way the Wall Street Journal put it, "Palantirians are the new Googlers. What search algorithms were to the 1990s, big data is today: a game changer. " The major change in the game for JP Morgan, according to the Financial Times, is that it is set to turn the software on ferreting out possible risks among its 250,000 employees.
While Palantir’s data tends to be associated with sifting through the trail left by the bad guys, it also can track the good guys who want to help. After Hurricane Sandy devastated the Rockaway area of New York, the US military veterans who contribute to Team Rubicon teamed with Palantir’s multi-source mapping capabilities to organize disaster response efforts.
As the above video shows, before the system was put in place, people who came to help had to rely on pen and paper to mark down where to go. With thousands of volunteers to manage, the low-tech approach proved far from efficient. As reported in Businessweek, Brendan Kraft, one of the volunteers, observed that there was a serious problem with mobilizing their resources: "You’d send people out to check a house, and you had no idea where they were or what they needed until they came back."
Saving lives and tracking terrorists
The solution arrived in a bus, which served as the intelligence center with Palantir software on laptops. Combining their maps with status updates via smartphones, dispatchers were able to keep tabs on volunteer activity and check off what was needed. As Alec Augustine-Marceil, a Palantir Technologies embedded analyst, observed on the Team Rubicon website, "I was in Afghanistan a week ago using this same system to track terrorists."
As a society we are very far from beating swords into ploughshares, but at least we can turn the big data tactics used by the military toward humanitarian ends. Meanwhile, banks and other businesses will see how the same tools can serve their own ends by harnessing the power of their big data. Whether the "seeing stones" are used for good or evil is up to the one using them.
Ariella,
User Rank: Blogger 1/22/2013 | 10:06:09 AM
Re: a panacea at last?> @Saul of course, it would. You just inspired me to create a meme. I can't the picture to post here now, but you can find it here.
Ariella,
User Rank: Blogger 1/22/2013 | 9:57:09 AM
Re: Protect from the evil @Susan Thanks, of course the seeing stones in the hands of the wrong person proved dangerous, as did the rings. That's why some conclude that he was opposed to technology:
Tolkien claimed that he never stooped to allegory in his writings, but he did not deny "applicability." Thus, The Lord of the Rings can be read as his response to modernity, to the world of catastrophic wars, terrible weapons, and industrialization that Tolkien felt was destroying his beloved rural, Edwardian England (represented in his books by the hobbits' peaceful, if parochial, homeland of "the Shire"). And if Tolkien's One Ring represents technology, or humanity's hubristic capacity to tamper with nature, then the message is: Destroy it forever.
Some scholars see in Tolkien's strongly anti-technology views a powerful enviro-Luddite strain. In his 1997 book Defending Middle Earth: Tolkien, Myth, and Modernity, Patrick Curry treats Tolkien as a kind of Green movement precursor--a literary Lorax. "In all my works I take the part of trees as against all their enemies," Tolkien wrote in 1972. But there's more than just an admiration of nature in Tolkien; there's the converse, a deep distrust of all things "unnatural." When the wizard Saruman presumes to tinker with nature, the Ent Treebeard reacts by saying, "That would be a black evil!" The Jeremy Rifkins and Kirkpatrick Sales of the world--along with other opponents of human-genome research, cloning, and biotechnology--would find a kindred spirit in Tolkien. So, for that matter, would the Unabomber.
Oh. :( But. Oh. Okay. Does that mean that we have to give legalcio a new secret identity, or hide him to keep him safe?
should I take it as a given that this very message will have it's sentiment analysed?
Most like someone is out there in the watch. How much big data is generated by Big Data Republic? That's one of the next research topics that will become hot news. :D
Saul Sherry,
User Rank: Blogger 1/21/2013 | 7:40:29 AM
Re: a panacea at last?> @Susan, we never leave our community in the cold! Interesting to think though, what webcrawlers (apart from Google et al.) of course are out there now, pouring over what we discuss here - should I take it as a given that this very message will have it's sentiment analysed?
As far as humans exist there will be good and evil. Maybe one of the tricks is to protect the big data tactics used by the military from the hands of those who would use them for evil. The question is how.
"but I do know people in the agency. Now destroy this post..."
You should have activated the self-destruct function. :D Your post is now being tracked, saved, and analyzed. We all at BDR will deny we have ever interacted with you.
SharCo,
User Rank: Bit Player 1/20/2013 | 12:43:02 AM
Re: a panacea at last?> As long as it lets you skip over some red tape especially during times where you need to (in cases of emergency, for example)--then I'm all for it. No sorcery here; just pure technology. Terrorist applications, a big yes also, although as Saul mentioned, there might be some complications.
Ariella,
User Rank: Blogger 1/18/2013 | 11:23:23 AM
Re: a panacea at last?> @Saul exactly. The worst that can happen with Amazon's algorithms being off is that you find the suggestions irrelevant or too late because you've already decided to purchase something else. But being right in time is essential here as lives do depend on making the prediction about terrorist movements before they happen.
To save this item to your list of favorite Big Data Republic content so you can find it later in your Profile page, click the "Save It" button next to the item.
If you found this interesting or useful, please use the links to the services below to share it with other readers. You will need a free account with each service to share an item via that service.
![By Flickr user PartnerHund.com, cropped and color adjusted by Zanaq. [CC-BY-2.0], via Wikimedia Commons](http://img.deusm.com/bigdatarepublic/2013/01/257339/122621_271164.jpg)
Tweet This
