PadicoTM documentation

Documentation for PadicoTM-2026-01-21+1805f140ea794afbf38e0649a1de9d4793e65f4f (build runner-7cspzrfxd-project-25853-concurrent-0 Mon Apr 13 16:46:44 UTC 2026)

PadicoTM developer's guide

This documentation is about using PadicoTM in your codes. It is intended for developers only.

• documentation for PadicoTM Core
• List of All PadicoTM components supplied by PadicoTM
• List of All PadicoTM API supplied by PadicoTM
• List of All PadicoTM services supplied by PadicoTM
• tips about hacking in PadicoTM
• cheat sheet for topology model
• PadicoTM uses Puk for data structures, modules, and component model.
  • see Puk documentation
• PukABI is used for symbol interception.
  • see PukABI documentation

Component interfaces

Here is a list of important component interfaces in PadicoTM:

• PadicoControl: this is the interface for the control channel. The control channel is a communication channel that interconnects nodes to exchange control messages. Messages are written in XML and are given to the XML parser uppon receipt, thus no explicit receive is needed. It is used to exchange topology information, synchronization, or as last resort communication channel. It is assumed that all known nodes are reachable through the communication channel.
  • defined in PadicoControl.h
  • driver struct padico_control_driver_s
  • managed by module PadicoControl: control channel
  • component: Control_SysIO- control channel over SysIO implements PadicoControl over SysIO (sockets)
  • component: Control_Router- control channel over a predefined channel implements routing for PadicoControl so as to interconnect nodes with no direct connection
  • component: Control_SSH2- control channel over ssh tunnel. implements PadicoControl over an ssh tunnel. This is deprecated now that Control_SysIO supports SocketFactory.
  • Control_RPC in NewMadeleine implements PadicoControl over nmad RPCs.
  • Control_mini in NewMadeleine implements PadicoControl over NewMad_minidriver.
• PadicoBootstrap: this is the interface used to bootstrap PadicoControl connections. Bootstraping a connection means establish the connection without any prior connection used to exchange addresses. A component which exposes PadicoBootstrap interface will always expose PadicoControl at the same time.
  • defined in PadicoControl.h
  • driver struct padico_bootstrap_driver_s
  • component: Control_SysIO- control channel over SysIO is able to bootstrap
  • component: Control_SSH2- control channel over ssh tunnel. is able to bootstrap
  • ControlBootstrap: initialize control channel is the module that bootstraps connections upon init, using component that provides PadicoBootstrap interface. See ControlBootstrap: initialize control channel for configuration.
• PadicoConnector: this is the interface used to establish PadicoControl connections, assuming a control channel is already available to exchange addresses. This is used to establish a faster connection (on a faster network), when only a slow network (i.e. TCP/IP) is able to bootstrap.
  • defined in PadicoControl.h
  • driver struct padico_bootstrap_driver_s
  • component: Control_SysIO- control channel over SysIO exposes PadicoConnector interface
  • component: SecondaryConnector- use primary controlers as secondary is able to expose PadicoConnector interface on top of a component with only PadicoBootstrap. This is deprecated
  • Control_mini in NewMadeleine implements PadicoConnector on top of NewMad_minidriver
• ControlSelector: this is the interface used to choose which instance of component to use to send a control message towards a given node. This may look similar to NetSelector; the difference is that NetSelector returns a component assembly (that will be instantiated), which ControlSelector returns an existing instance.
  • defined in PadicoControl.h
  • driver struct padico_control_selector_driver_s
  • component: ControlSimpoleSelector- simple selector for the control channel offers a basic policy: return the existing instance of controler
  • component: ControlTwoStepSelector- selector setting up a direct offers a more advanced policy: try to establish a secondary faster connection if possible.
• NetSelector: this is the interface used to select the network to use. The component returns a component (or an assembly, as a composite) when requested for a host or a node.
  • defined in NetSelector.h
  • driver struct padico_netselector_driver_s
  • managed by module NetSelector- The PadicoTM network selector
  • component: NetSelector_inet_default- default selector used for interoperable connections. is the default selector used for hosts that are unknown to PadicoTM, and thus must be connected with an interoperable communication method.
  • component: NetSelector_besteffort- NetSelector with a best-effort policy uses a best-effort policy: use high-speed networks if available, then regular sockets, and the control channel as last resort. Itheavily relies on Topology- discover networks and nodes to compute the available links, and may be tuned to enable/disable rules.
  • component: NetSelector_basic- NetSelector with an explicit configuration file uses an explicit policy: the user defines rules with filters that apply to topology, and gives explicitely which component to use. It is configured through an XML configuration file. Whithout a proper configuration file containing the actual hosts, it does nothing.
• SocketFactory: this interface is used to create sockets of different kinds. It is only involved for socket creation, not after they are connected.
  • defined in SocketFactory.h
  • driver struct padico_socketfactory_s
  • component: SocketFactory_plain- Socket factory which builds plain TCP sockets. implements SocketFactory for regular TCP/IP sockets
  • component: SocketFactory_Reverse_Forward- Socket factory which builds a socket in makes connection in reverse direction to cross firewalls that filter connections only in one direction
  • component: SocketFactory_Splicing- socket factory that performs TCP splicing uses TCP splicing, i.e. simultaneous connections, to cross statefull firewalls that filters connections in both directions.
  • component: SocketFactory_SSH_Tunnel- socket factory for SSH tunnels establish an SSH tunnel
  • component: SocketFactory_Reversed_SSH_Tunnel- socket factory for reversed SSH tunnels established an SSH tunnel in the revrse direction
  • component: SocketFactory_SSH_Netcat- socket factory for communication through netcat over SSH console establishes an SSH tunnel and uses Netcat at destination to reach another node than the ssh host
  • component: SocketFactory_SSH2- socket factory for communication through SSH2 tunnels establishes an SSH2 tunnel
  • component: SocketFactory_LibSSH2- socket factory for communication through SSH2 tunnels uses libssh2 to establish SSH tunnels. This is more robust than other methods since it uses directly a library designed for this usage, instead of forking a plain ssh binary
  • component: SOCKS4_client- socket factory of a SOCKS4 proxy. establishes a connection through a SOCKS4 proxy.
• PacketFilter: this interface is designed to interpose a filter that process all packets going through
  • defined in PacketFilter.h
  • driver struct packetfilter_driver_s
  • component: PacketFilter_ID- a packet filter that does nothing (identity function) is a filter that does nothing, used for testing
  • component: PacketFilter_ZIP- a packet filter that performs ZIP compression applies ZIP compression
  • component: PacketFilter_LZO- a packet filter that performs realtime LZO compression applies LZO compression
  • component: PacketFilter_ZSTD - a packet filter that performs ZSTD compression applies ZSTD compression
  • component: PacketFilter_BZ2- a packet filter that performs bzip2 compression applies BZ2 compression
  • component: PacketFilter_LZ4- a packet filter that performs LZ4 compression applies LZ4 compression
  • component: VLink_PacketFilter- VLink over VLink with pluggable packet filtering applies a PacketFilter on packets from a VLink connection
  • PMIx- Slurm support through PMIx uses PacketFilter to compress topology and addresses exchange.
• VLink: this interface is designed to mimic the semantics of the socket API. It can be used easily through the VIO- virtual I/O that mimics stdio, or transparently without modifying the code when using PukABI.
  • defined in VLink-API.h
  • driver padico_vlink_driver_s
  • component: VLink_Packet- VLink over Padico Simple Packets (PSP) implements VLink on top of PadicoSimplePackets. It enables to have socket-based software use high-speed networks
  • component: VLink_SocketFactory- VLink over socket factories implements VLink interface on top of SocketFactory, and thus plain sockets and the variety of tunnels and proxies.
  • component: VLink_PacketFilter- VLink over VLink with pluggable packet filtering applies a PacketFilter on the stream of packets of a VLink connection. It must be used on top of another component that provides VLink.
  • component: VLink_ID- VLink over VLink dummy component is a VLink filter that does nothing. It is used for testing.
  • component: VLink_fwd- VLink over VLink "through-a-node" component establishes a VLink connection through a third node that perform forwarding
  • component: VLink_selector- automatic server-side VLink selector allows a server-side socket to listen to both interoperable plain socket and a PadicoTM-specific component at the same time, so as to accept both types of connections.
  • component: ParallelStreams- use multiple sockets to improve bandwidth uses multiple socket connections at the same time to improve bandwidth on long connections, similar to what GridFTP does.
• PadicoSimplePackets: this is a simple low-level packet-based communication interface, with implicit connection and implicit receive (active messages). When a message is received, a callback is called to process it immediately. It is designed to cooperate well with multi-threaded environments.
  • defined in PSP.h
  • driver struct padico_psp_driver_s
  • managed by module PSP- Padico Simple Packets.
  • component: PSP_Control- PSP over control channel. implements PSP over PadicoControl
  • component: PSP_SocketFactory- PSP over SocketFactory. implements PSP over sockets
  • component: PSP_Shm- inter-process shared memory communication. implements PSP over shared memory for intra-node communication
  • component: PSP_InfinibandVerbs- PSP over InfiniBand implements PSP over InfiniBand (ibverbs)
  • component: PSP_MX- Direct messaging over Myrinet Express implements PSP over Myrinet MX (obsolete)

All components: All PadicoTM components

Programming interfaces (API)

APIs are plain C interfaces, without the need for a component indirection. They are provided by a module. A module that wants to use them should have an explicit dependancy on the module (through padico-mkmod --requires).

Here is a list of important APIs in PadicoTM:

• PadicoTM Core, the interface of the core
• Topology- discover networks and nodes to get information on networks and nodes. See the cheat sheet for topology model
• PadicoControl: control channel to exchange control messages
• AddrDB- address exchange to exchange addresses
• NetSelector- The PadicoTM network selector a front-end for NetSelector
• PSP- Padico Simple Packets. for packet-based high-performance messaging
• VIO- PadicoTM virtual I/O for socket-like communications
• SysIO: shared polling for file descriptors the low-level API used for a shared polling of file descriptors
• PadicoBootLib to use PadicoTM as a library
• PMIx- Slurm support through PMIx to interface with slurm through PMIx
• Connector- front-end to PadicoConnector, to explicitely a front-end for PadicoConnector

All APIs: All PadicoTM API

Services

Services are modules that provide a service by themselves. They do not provide any API.

Here is a list of some services:

• ControlBootstrap: initialize control channel to establish bootstrap connection at init. Parameters to load it are added automatically by padico-launch, unless given the -nodefault flag.
• InfinibandVerbs- detect InfiniBand devices and populate network topology. detects InfiniBand networks. It is loaded automatically by Topology if a /dev/infiniband is detected.
• MemMonitor- memory profiling is a service that dumps periodically memory load. It can be loaded by giving the parameter -iload-MemMonitor to padico-launch
• ProfMonitor- dumps profiling variable periodically is a service that dumps periodically Puk profiling variables. It can be loaded by giving the parameter -iload-ProfMonitor to padico-launch
• Console- interactive console plugged on local tty is a service that opens a console on the current tty to interact with PadicoTM. It is usually loaded as the main program, i.e. padico-launch Console.
• RemoteControl- server-side for XML/socket remote control is a service to control PadicoTM through a GUI (obsolete). It is automatically loaded when PadicoTM is launched by the GUI.
• CORBAGatekeeper, Service: SOAP-Gatekeeper – server-side for SOAP remote control, Tube_GateKeeper are services to control PadicoTM through various communication methods (obsolete)
• Sh_driver- Bourne shell scripts support in PadicoTM is a service that add a Puk driver to execute shell scripts as modules. It is loaded on-demand, when a loaded module uses sh driver.
• LegacyBinary- support for legacy binary codes to preload PadicoTM into legacy binaries. It is loaded explicitely by padico-launch.

All services: All PadicoTM services

Further reading

• PadicoTM homepage