A Game of Data and GraphQL

  • Published on
    18-Mar-2018

  • View
    222

  • Download
    3

DESCRIPTION

A GAME OF DATA AND GRAPHQL Michael Hunger, Head of Developer Relations, Neo4j (@mesirii) GraphQL Meetup Berlin, Aug 24 2017 Looking for GoT Data AN API OF ICE AND FIRE https://anapioficeandfire.com/About…

Transcript

A GAME OF DATA AND GRAPHQL Michael Hunger, Head of Developer Relations, Neo4j (@mesirii) GraphQL Meetup Berlin, Aug 24 2017 Looking for GoT Data AN API OF ICE AND FIRE https://anapioficeandfire.com/About https://github.com/joakimskoog/AnApiOfIceAndFire An API of Ice and Fire ⢠Data sourced from Wiki of Ice And Fire (AWOIAF) ⢠Well documented .Net powered API ⢠Data on â Books (5) â Characters (2400) â Houses (444) ⢠100 entries per page An API of Ice and Fire â https://www.anapioficeandfire.com/api/characters/1303 https://www.anapioficeandfire.com/api/characters/1303 GRAPHQL Quick Graphql Schema type Seat { name: String! houses: [House] @relation(name:"SEAT_OF") } type Region { name: String! houses: [House] @relation(name:"IN_REGION", direction:IN) } type House { id: ID! name: String! founded: String titles: [String] ancestralWeapons: [String] coatOfArms: String words: String seats: [Seat] @relation(name:"SEAT_OF", direction:IN) region: Region @relation(name:"IN_REGION") leader: Person @relation(name:"LED_BY") founder: Person @relation(name:"FOUNDED_BY") allies: [House] @relation(name:"ALLIED_WITH", direction:IN) follows: House @relation(name:"SWORN_TO") followers: [House] @relation(name:"SWORN_TO", direction:IN) heir: [Person] @relation(name:"HEIR_TO", direction:IN) } type Person { id: ID! name: String! aliases: [String] books: [Int] tvSeries: [String] playedBy: [String] isFemale: Boolean culture: String died: String titles: [String] founded: [House] @relation(name:"FOUNDED_BY", direction:IN) leads: [House] @relation(name:"LED_BY", direction:IN) inherits: [House] @relation(name:"HEIR_TO") spouse: [Person] @relation(name:"SPOUSE", direction:BOTH) parents: [Person] @relation(name:"PARENT_OF", direction:IN) children: [Person] @relation(name:"PARENT_OF") houses: [House] @relation(name:"ALLIED_WITH") } SPIN UP AN ENDPOINT $ npm install âg neo4j-graphql-cli $ neo4j-graphql got-schema.graphql # configure graphql-cli $ npm install âg graphql-cli $ graphql init # add Auth Headers Visualize Schema call graphql.schema(); Visualize Schema $ graphql voyager LOADING THE DATA LOAD VIA API ⢠Iterate over batches of pages ⢠Data cleanup ⢠Replace URLs with IDs ⢠Throttle ⢠INSERT COMPLEX SCRIPT HERE unwind range(1,43) as page call apoc.util.sleep(1) with page, 'characters' as type call apoc.load.jsonArray('https://www.anapioficeandfire.com/api/'+type+'?pageSize=50&page='+page) yield value with apoc.convert.toMap(value) as data MERGE (p:Person {id:split(data.url,"/")[-1]}) SET p += apoc.map.clean(data, ['allegiances','books','father','spouse','mother'],['',[''],[]]), p.books = [b in data.books | split(b,'/')[-1]], p.name = colaesce(p.name,head(p.aliases)) FOREACH (a in data.allegiances | MERGE (h:House {id:split(a,'/')[-1]}) MERGE (p)-[:SWORN_TO]->(h)) FOREACH (f in case coalesce(data.father,"") when "" then [] else [data.father] end | MERGE (o:Person {id:split(f,'/')[-1]}) MERGE (o)-[:PARENT_OF {type:'father'}]->(p)) FOREACH (f in case coalesce(data.mother,"") when "" then [] else [data.mother] end | MERGE (o:Person {id:split(f,'/')[-1]}) MERGE (o)-[:PARENT_OF {type:'mother'}]->(p)) FOREACH (f in case coalesce(data.spouse,"") when "" then [] else [data.spouse] end | MERGE (o:Person {id:split(f,'/')[-1]}) MERGE (o)-[:SPOUSE]-(p)) return p.id, p.name; BETTER! FULL JSON DATA ⢠Full JSON data in Github Repository ⢠Load all characters and houses in one go ⢠No URL conversion ⢠But lowercasing keys ⢠Offline and much faster call apoc.load.jsonArray('https://raw.githubusercontent.com/joakimskoog/AnApiOfIceAndFire/master/data/houses.json') yield value with apoc.convert.toMap(value) as data with apoc.map.clean(data, [],['',[''],[],null]) as data with apoc.map.fromPairs([k in keys(data) | [toLower(substring(k,0,1))+substring(k,1,length(k)), data[k]]]) as data MERGE (h:House {id:data.id}) SET h += apoc.map.clean(data, ['overlord','swornMembers','currentLord','heir','founder','cadetBranches'],['',[''],[],null]) FOREACH (id in data.swornMembers | MERGE (o:Person {id:id}) MERGE (o)-[:ALLIED_WITH]->(h)) FOREACH (s in data.seats | MERGE (seat:Seat {name:s}) MERGE (seat)-[:SEAT_OF]->(h)) FOREACH (id in data.cadetBranches | MERGE (b:House {id:id}) MERGE (b)-[:BRANCH_OF]->(h)) FOREACH (id in case data.overlord when null then [] else [data.overlord] end | MERGE (o:House {id:id}) MERGE (h)-[:SWORN_TO]->(o)) FOREACH (id in case data.currentLord when null then [] else [data.currentLord] end | MERGE (o:Person {id:id}) MERGE (h)-[:LED_BY]->(o)) FOREACH (id in case data.founder when null then [] else [data.founder] end | MERGE (o:Person {id:id}) MERGE (h)-[:FOUNDED_BY]->(o)) FOREACH (id in case data.heir when null then [] else [data.heir] end | MERGE (o:Person {id:id}) MERGE (o)-[:HEIR_TO]->(h)) FOREACH (r in case data.region when null then [] else [data.region] end | MERGE (o:Region {name:r}) MERGE (h)-[:IN_REGION]->(o)) return h.id, h.name; ITS A GRAPH ITS A GRAPH QuERY Data with Cypher MISSING DATA !? MATCH (p:Person) WHERE size(p.tvSeries) > 1 AND NOT exists((p)-[:PARENT_OF]-()) RETURN p LIMIT 10; ⢠Walder ⢠The waif ⢠High Septon ⢠Margaery Tyrell ⢠Tywin Lannister ⢠Unella ⢠Aemon Targaryen ⢠Alliser Thorne ⢠Arya Stark ⢠Asha Greyjoy QuERY Data WITH Graphql RESOURCES Code and instructions for todays presentation ⢠GitHub â https://github.com/neo4j-examples/game-of-thrones ⢠Medium Post â medium.com/@mesirii/a-game-of-data-and-graphql-97ee2ca297ce ⢠Graph of Thrones 7 week contest â https://neo4j.com/blog/graph-of-thrones/ ⢠An API of Ice and Fire â https://github.com/joakimskoog/AnApiOfIceAndFire â https://anapioficeandfire.com/About ⢠GraphQL CLI Load â https://npmjs.org/package/graphql-cli-load https://github.com/neo4j-examples/game-of-thrones mailto:medium.com/@mesirii/a-game-of-data-and-graphql-97ee2ca297ce https://neo4j.com/blog/graph-of-thrones/ https://github.com/joakimskoog/AnApiOfIceAndFire https://anapioficeandfire.com/About https://npmjs.org/package/graphql-cli-load Neo 4J GraphQL ⢠http://neo4j.com/developer/graphql ⢠https://npmjs.org/package/neo4j-graphql-cli ⢠http://communitygraph.org/graphql ⢠http://github.com/neo4j-graphql â graphiql4all â graphql-cli-load â neoj4-graphql-cli â community graph â ... http://neo4j.com/developer/graphql https://npmjs.org/package/neo4j-graphql-cli http://communitygraph.org/graphql http://github.com/neo4j-graphql More Fun with Graphs and Game of Thrones ⢠Character Interactions â https://networkofthrones.wordpress.com/ â :play https://guides.neo4j.com/got ⢠Graph Analytics of Character Interactions â http://www.lyonwj.com/2016/06/26/graph-of-thrones-neo4j- social-network-analysis/ ⢠NLP on the book texts â https://graphaware.com/neo4j/2017/07/24/reverse- engineering-book-stories-nlp.html ⢠The Maths of Game of Thrones â https://anthonybonato.com/2016/04/13/the-mathematics-of- game-of-thrones/ ⢠Kaggle GoT Data (Battles) â https://tbgraph.wordpress.com/?s=Game+of+Thrones https://networkofthrones.wordpress.com/ http://www.lyonwj.com/2016/06/26/graph-of-thrones-neo4j-social-network-analysis/ https://graphaware.com/neo4j/2017/07/24/reverse-engineering-book-stories-nlp.html https://anthonybonato.com/2016/04/13/the-mathematics-of-game-of-thrones/ QUESTIONS @mesirii | neo4j.com/slack