Tensorflow architecture

Architecture

- tensorflow architecture를

tensorflow docs _ architecture와 [생각의 웹] 텐서플로우 소스 구조 분석 (TensorFlow Internal)를 참고하여 공부하였다.

• Overview

  

  Client : Defines dataflow graph / Initiates session
  C API : Separates user level code in different languages from core runtime
  Distributed Master : Prunes graph / Distributes them to worker services
  Worker Services : Schedule to appropriate to the available hardware
  Kernel : Perform the computation

• Client

  - Tensorflow supports multiple client languages (typically Python , C++). Because the C API at the bottom of client, seperates user-level code in different language from core runtime.

  

  - Client defines data flow graph, and create session that sends graph definition to distributed master as tf.GraphDef protocol buffer

  

• Distributed Master

  - Distributed master prunes the graph and partitions the graph to obtain graph pieces for each participating device. Groups the model parameters in order to place them together on the parameter server.

  

    Where graph edges are cut by the partition, the distributed master inserts send and receive nodes to pass information between the distributed tasks

  

• Worker Service

  - Recieves subgraph from Distribute maste.
  - schedules the execution of the kernels for the operations that comprise a local subgraph, and mediates direct communication between tasks. Dispatches kernels to local devices, runs kernels in parallel devices such as CPUs, GPUs.
  - Speciallized Send / Recv operations. "cudaMemcpyAsync()" API is for transfers between local CPU and GPU devices.
  - protocols tensorflow uses to transfer
  gRPC over TCP RDMA over Converged Ethernet

  

• Kernel