TensorFlow is an open source software library that utilises data flow graphs for numerical computation. It plays a key role in research and production at Google products, and often replaces DistBelief,  its predecessor. Before being made open source, Tensor flow was developed by the Google Brain team with the aim to just use it internally.

Something that makes TensorFlow unique is that it gives you the ability to perform partial subgraph computation. This enables the partitioning of a neural network, making distributing training a realistic possibility.  Subgraph computation then allows TensorFlow to support Model Parallelism. State nodes in the graph support Data Parallelism.