Class/Object

org.graphframes

GraphFrame

Related Docs: object GraphFrame | package graphframes

Permalink

class GraphFrame extends Logging with Serializable

A representation of a graph using DataFrames.

Linear Supertypes
Serializable, Serializable, Logging, LazyLogging, com.typesafe.scalalogging.Logging, AnyRef, Any
Ordering
  1. Grouped
  2. Alphabetic
  3. By Inheritance
Inherited
  1. GraphFrame
  2. Serializable
  3. Serializable
  4. Logging
  5. LazyLogging
  6. Logging
  7. AnyRef
  8. Any
Implicitly
  1. by any2stringadd
  2. by StringFormat
  3. by Ensuring
  4. by ArrowAssoc
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new GraphFrame()

    Permalink

    Default constructor is provided to support serialization

    Default constructor is provided to support serialization

    Attributes
    protected

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. def +(other: String): String

    Permalink
    Implicit information
    This member is added by an implicit conversion from GraphFrame to any2stringadd[GraphFrame] performed by method any2stringadd in scala.Predef.
    Definition Classes
    any2stringadd
  4. def ->[B](y: B): (GraphFrame, B)

    Permalink
    Implicit information
    This member is added by an implicit conversion from GraphFrame to ArrowAssoc[GraphFrame] performed by method ArrowAssoc in scala.Predef.
    Definition Classes
    ArrowAssoc
    Annotations
    @inline()
  5. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  6. def aggregateMessages: AggregateMessages

    Permalink

    This is a primitive for implementing graph algorithms.

    This is a primitive for implementing graph algorithms. This method aggregates values from the neighboring edges and vertices of each vertex. See AggregateMessages for detailed documentation.

  7. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  8. def bfs: BFS

    Permalink

    Breadth-first search (BFS)

    Breadth-first search (BFS)

    Refer to the documentation of org.graphframes.lib.BFS for the description of the output.

  9. def cache(): GraphFrame.this.type

    Permalink

    Persist the dataframe representation of vertices and edges of the graph with the default storage level.

  10. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  11. def connectedComponents: ConnectedComponents

    Permalink

    Connected component algorithm.

    Connected component algorithm.

    See org.graphframes.lib.ConnectedComponents for more details.

  12. lazy val degrees: DataFrame

    Permalink

    The degree of each vertex in the graph, returned as a DataFrame with two columns:

    The degree of each vertex in the graph, returned as a DataFrame with two columns:

    • GraphFrame.ID the ID of the vertex
    • 'degree' (integer) the degree of the vertex Note that vertices with 0 edges are not returned in the result.
  13. def edgeColumnMap: Map[String, Int]

    Permalink

    Version of edgeColumns which maps column names to indices in the Rows.

  14. def edgeColumns: Array[String]

    Permalink

    The vertex names in the vertices DataFrame, in order.

    The vertex names in the vertices DataFrame, in order.

    Helper method for toGraphX which specifies the schema of edge attributes. The edge attributes of the returned edges are given as a Row, and this method defines the column ordering in that Row.

  15. def edges: DataFrame

    Permalink

    The dataframe representation of the edges of the graph.

    The dataframe representation of the edges of the graph.

    It contains two columns called GraphFrame.SRC and GraphFrame.DST that contain the ids of the source vertex and the destination vertex of each edge, respectively. It may also contain various other columns with user-defined attributes for each edge.

    For symmetric graphs, both pairs src -> dst and dst -> src are present with the same attributes for each pair.

    The order of the columns is available in edgeColumns.

  16. def ensuring(cond: (GraphFrame) ⇒ Boolean, msg: ⇒ Any): GraphFrame

    Permalink
    Implicit information
    This member is added by an implicit conversion from GraphFrame to Ensuring[GraphFrame] performed by method Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  17. def ensuring(cond: (GraphFrame) ⇒ Boolean): GraphFrame

    Permalink
    Implicit information
    This member is added by an implicit conversion from GraphFrame to Ensuring[GraphFrame] performed by method Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  18. def ensuring(cond: Boolean, msg: ⇒ Any): GraphFrame

    Permalink
    Implicit information
    This member is added by an implicit conversion from GraphFrame to Ensuring[GraphFrame] performed by method Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  19. def ensuring(cond: Boolean): GraphFrame

    Permalink
    Implicit information
    This member is added by an implicit conversion from GraphFrame to Ensuring[GraphFrame] performed by method Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  20. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  21. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  22. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  23. def find(pattern: String): DataFrame

    Permalink

    Motif finding: Searching the graph for structural patterns

    Motif finding: Searching the graph for structural patterns

    Motif finding uses a simple Domain-Specific Language (DSL) for expressing structural queries. For example, graph.find("(a)-[e]->(b); (b)-[e2]->(a)") will search for pairs of vertices a,b connected by edges in both directions. It will return a DataFrame of all such structures in the graph, with columns for each of the named elements (vertices or edges) in the motif. In this case, the returned columns will be in order of the pattern: "a, e, b, e2."

    DSL for expressing structural patterns:

    • The basic unit of a pattern is an edge. For example, "(a)-[e]->(b)" expresses an edge e from vertex a to vertex b. Note that vertices are denoted by parentheses (a), while edges are denoted by square brackets [e].
    • A pattern is expressed as a union of edges. Edge patterns can be joined with semicolons. Motif "(a)-[e]->(b); (b)-[e2]->(c)" specifies two edges from a to b to c.
    • Within a pattern, names can be assigned to vertices and edges. For example, "(a)-[e]->(b)" has three named elements: vertices a,b and edge e. These names serve two purposes:
      • The names can identify common elements among edges. For example, "(a)-[e]->(b); (b)-[e2]->(c)" specifies that the same vertex b is the destination of edge e and source of edge e2.
      • The names are used as column names in the result DataFrame. If a motif contains named vertex a, then the result DataFrame will contain a column "a" which is a StructType with sub-fields equivalent to the schema (columns) of GraphFrame.vertices. Similarly, an edge e in a motif will produce a column "e" in the result DataFrame with sub-fields equivalent to the schema (columns) of GraphFrame.edges.
    • It is acceptable to omit names for vertices or edges in motifs when not needed. E.g., "(a)-[]->(b)" expresses an edge between vertices a,b but does not assign a name to the edge. There will be no column for the anonymous edge in the result DataFrame. Similarly, "(a)-[e]->()" indicates an out-edge of vertex a but does not name the destination vertex.
    • An edge can be negated to indicate that the edge should *not* be present in the graph. E.g., "(a)-[]->(b); !(b)-[]->(a)" finds edges from a to b for which there is *no* edge from b to a.

    More complex queries, such as queries which operate on vertex or edge attributes, can be expressed by applying filters to the result DataFrame.

    pattern

    Pattern specifying a motif to search for.

    returns

    DataFrame containing all instances of the motif.

  24. def formatted(fmtstr: String): String

    Permalink
    Implicit information
    This member is added by an implicit conversion from GraphFrame to StringFormat[GraphFrame] performed by method StringFormat in scala.Predef.
    Definition Classes
    StringFormat
    Annotations
    @inline()
  25. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  26. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  27. lazy val inDegrees: DataFrame

    Permalink

    The in-degree of each vertex in the graph, returned as a DataFame with two columns:

    The in-degree of each vertex in the graph, returned as a DataFame with two columns:

    • GraphFrame.ID the ID of the vertex "- "inDegree" (int) storing the in-degree of the vertex Note that vertices with 0 in-edges are not returned in the result.
  28. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  29. def labelPropagation: LabelPropagation

    Permalink

    Label propagation algorithm.

    Label propagation algorithm.

    See org.graphframes.lib.LabelPropagation for more details.

  30. def logDebug(s: String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  31. def logInfo(s: String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  32. def logTrace(s: String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  33. lazy val logger: Logger

    Permalink
    Attributes
    protected
    Definition Classes
    LazyLogging → Logging
  34. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  35. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  36. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  37. lazy val outDegrees: DataFrame

    Permalink

    The out-degree of each vertex in the graph, returned as a DataFrame with two columns:

    The out-degree of each vertex in the graph, returned as a DataFrame with two columns:

    • GraphFrame.ID the ID of the vertex
    • "outDegree" (integer) storing the out-degree of the vertex Note that vertices with 0 out-edges are not returned in the result.
  38. def pageRank: PageRank

    Permalink

    PageRank algorithm.

    PageRank algorithm.

    See org.graphframes.lib.PageRank for more details.

  39. def parallelPersonalizedPageRank: ParallelPersonalizedPageRank

    Permalink

    Parallel personalized PageRank algorithm.

    Parallel personalized PageRank algorithm.

    See org.graphframes.lib.ParallelPersonalizedPageRank for more details.

  40. def persist(newLevel: StorageLevel): GraphFrame.this.type

    Permalink

    Persist the dataframe representation of vertices and edges of the graph with the given storage level.

    Persist the dataframe representation of vertices and edges of the graph with the given storage level.

    newLevel

    One of: MEMORY_ONLY, MEMORY_AND_DISK, MEMORY_ONLY_SER, MEMORY_AND_DISK_SER, DISK_ONLY, MEMORY_ONLY_2, MEMORY_AND_DISK_2, etc..

  41. def persist(): GraphFrame.this.type

    Permalink

    Persist the dataframe representation of vertices and edges of the graph with the default storage level.

  42. def shortestPaths: ShortestPaths

    Permalink

    Shortest paths algorithm.

    Shortest paths algorithm.

    See org.graphframes.lib.ShortestPaths for more details.

  43. def stronglyConnectedComponents: StronglyConnectedComponents

    Permalink

    Strongly connected components algorithm.

    Strongly connected components algorithm.

    See org.graphframes.lib.StronglyConnectedComponents for more details.

  44. def svdPlusPlus: SVDPlusPlus

    Permalink

    SVD++ algorithm.

    SVD++ algorithm.

    See org.graphframes.lib.SVDPlusPlus for more details.

  45. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  46. def toGraphX: Graph[Row, Row]

    Permalink

    Converts this GraphFrame instance to a GraphX Graph.

    Converts this GraphFrame instance to a GraphX Graph. Vertex and edge attributes are the original rows in vertices and edges, respectively.

    Note that vertex (and edge) attributes include vertex IDs (and source, destination IDs) in order to support non-Long vertex IDs. If the vertex IDs are not convertible to Long values, then the values are indexed in order to generate corresponding Long vertex IDs (which is an expensive operation).

    The column ordering of the returned Graph vertex and edge attributes are specified by vertexColumns and edgeColumns, respectively.

  47. def toString(): String

    Permalink
    Definition Classes
    GraphFrame → AnyRef → Any
  48. def triangleCount: TriangleCount

    Permalink

    Triangle count algorithm.

    Triangle count algorithm.

    See org.graphframes.lib.TriangleCount for more details.

  49. lazy val triplets: DataFrame

    Permalink

    Returns triplets: (source vertex)-[edge]->(destination vertex) for all edges in the graph.

    Returns triplets: (source vertex)-[edge]->(destination vertex) for all edges in the graph. The DataFrame returned has 3 columns, with names: GraphFrame.SRC, GraphFrame.EDGE, and GraphFrame.DST. The 2 vertex columns have schema matching GraphFrame.vertices, and the edge column has a schema matching GraphFrame.edges.

  50. def unpersist(blocking: Boolean): GraphFrame.this.type

    Permalink

    Mark the dataframe representation of vertices and edges of the graph as non-persistent, and remove all blocks for it from memory and disk.

    Mark the dataframe representation of vertices and edges of the graph as non-persistent, and remove all blocks for it from memory and disk.

    blocking

    Whether to block until all blocks are deleted.

  51. def unpersist(): GraphFrame.this.type

    Permalink

    Mark the dataframe representation of vertices and edges of the graph as non-persistent, and remove all blocks for it from memory and disk.

  52. def vertexColumnMap: Map[String, Int]

    Permalink

    Version of vertexColumns which maps column names to indices in the Rows.

  53. def vertexColumns: Array[String]

    Permalink

    The column names in the vertices DataFrame, in order.

    The column names in the vertices DataFrame, in order.

    Helper method for toGraphX which specifies the schema of vertex attributes. The vertex attributes of the returned Graph are given as a Row, and this method defines the column ordering in that Row.

  54. def vertices: DataFrame

    Permalink

    The dataframe representation of the vertices of the graph.

    The dataframe representation of the vertices of the graph.

    It contains a column called GraphFrame.ID with the id of the vertex, and various other user-defined attributes with other attributes.

    The order of the columns is available in vertexColumns.

  55. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  56. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  57. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  58. def [B](y: B): (GraphFrame, B)

    Permalink
    Implicit information
    This member is added by an implicit conversion from GraphFrame to ArrowAssoc[GraphFrame] performed by method ArrowAssoc in scala.Predef.
    Definition Classes
    ArrowAssoc

Inherited from Serializable

Inherited from Serializable

Inherited from Logging

Inherited from LazyLogging

Inherited from com.typesafe.scalalogging.Logging

Inherited from AnyRef

Inherited from Any

Inherited by implicit conversion any2stringadd from GraphFrame to any2stringadd[GraphFrame]

Inherited by implicit conversion StringFormat from GraphFrame to StringFormat[GraphFrame]

Inherited by implicit conversion Ensuring from GraphFrame to Ensuring[GraphFrame]

Inherited by implicit conversion ArrowAssoc from GraphFrame to ArrowAssoc[GraphFrame]

Conversions

Graph topology

Motif finding

Standard graph algorithms

Structure information

Ungrouped