TSDuck sample usages

TSDuck sample usages

TSDuck sample usages

TSP, the transport stream processor

The most important TSDuck tool is tsp, the transport stream processor. It is a flexible transport stream processing framework. Running tsp is a combination of elementary processing operations using plugins.

In each invocation of tsp, there are:

This is illustrated in the diagram below:

The currently existing input and output plugins can work on TS files, UDP/IP (multicast or unicast), HTTP, HLS, SRT, DVB/ATSC/ISDB sources using cheap tuners, professional Dektec modulators and ASI devices, HiDes modulators.

MPE injection and extraction

This example describes a test bed or demo infrastructure for MPE injection and MPE extraction.

The network infrastructure is illustrated in the diagram below.

In network 1, a media server multicasts a transport stream on address 224.250.250.1, port 9000. We want to encapsulate this UDP multicast stream in an existing transport stream using MPE. This existing TS is a live satellite feed which is received using a DVB tuner.

We also change the multicast destination address for the UDP stream to 230.2.3.4, port 7000, in the MPE-encapsulated datagrams. There is no particular reason for this, we just illustrate the feasibility.

The resulting transport stream with embedded MPE is then broadcast. Here, the broadcast network is a Dektec modulator, followed by another computer using a DVB tuner.

This computer is connected to a second network. Another instance of tsp extracts the datagrams from the MPE stream and multicasts them on its network using the modified destination address.

The first tsp command adds two services in the existing transport stream. One service carries the IP/MAC Notification Table (INT). The other one carries the MPE stream. We create a fully DVB-conformant signalization for these two services.

The second tsp command extracts the MPE stream from the received TS and forwards all decapsulated UDP multicast datagrams on the local network.

This example is fully described in the TSDuck User's Guide with all details and corresponding commands.

Merging and forking transport streams

Specific plugins such as merge and fork respectively combine and duplicate transport streams. They are designed to route transport streams from and to other applications. When the "other" application is another instance of tsp, we can create complex processing graphs.

This is illustrated in the diagram below.

In addition to tsp, the command tsswitch can be used to switch inputs between a set of transport streams using the same input and output plugins as tsp. Switching between inputs can be performed automatically at the end of each input or through a remote control application. The remote control protocol is extremely simple, based on text messages in UDP datagrams.

This is illustrated in the diagram below.

The remote control can be performed using simple command lines. The bash shell provides an easy way to redirect output to an UDP message. The following sample commands send UDP messages on port 4444 to system 127.0.0.1 (the local host). This is the easiest way to use the tsswitch remote control.

echo >/dev/udp/127.0.0.1/4444 2
echo >/dev/udp/127.0.0.1/4444 next

A CAS test bed

The following diagram illustrates a real-life example. To test a Conditional Access System and signalization, tsp is used to create a new transport stream from a live source. One real TS is received as input using a cheap USB or PCI DVB-T tuner. This TS and all its services are renamed, creating a "new TS" in the network. To test the CAS, tsp interacts with a real ECMG and a real EMMG using the standard DVB SimulCrypt protocols. One service, initially in the clear, is scrambled. tsp generates the control words, scrambles the content and inserts ECM's and EMM's. Finally, a set-top box receives all live transponders plus the new one we create.

Everything is done in one single tsp process. The command is the following. For clarity, the details such as tuning parameters, access criteria, IP addresses and ports, are represented as environment variables.

tsp -I dvb -u $UHF_INPUT                                      \
    -P tsrename -t 9 -a                                       \
    -P svrename direct8 -i 0x0901 -l 41 -n "Direct 8 Test"    \
    -P svrename bfmtv -i 0x0903 -l 42 -n "BFM TV Test"        \
    -P svrename 'i>tele' -i 0x0904 -l 43 -n "i>TELE Test"     \
    -P svrename virgin17 -i 0x0905 -l 44 -n "Virgin 17 Test"  \
    -P svrename gulli -i 0x0906 -l 45 -n "Gulli Test"         \
    -P svrename france4 -i 0x0907 -l 46 -n "France 4 Test"    \
    -P svrename 0x02FF -i 0x09FF                              \
    -P scrambler GulliTest -e $ECMG -s $SUPER_CAS_ID          \
         -p $PMT_CADESC_PRIVATE -a $AC                        \
         -b $ECM_BITRATE --pid $ECM_PID                       \
    -P cat -c -a $CAS_ID/$EMM_PID/$CAT_CADESC_PRIVATE         \
    -P datainject -r -s $MUX_SERVER_PORT                      \
          -b $EMM_MAX_BITRATE -p $EMM_PID                     \
    -O dektec -u $UHF_OUTPUT --convolution 2/3 --guard 1/32

This command is complex but rational. The option -I on the first line specifies the input plugin and its arguments. Here, the input plugin is dvb which drives DVB tuners. Each option -P specifies a packet processor plugin and its options. The plugins tsrename and svrename respectively rename the transport stream and a service. The plugin scrambler is a DVB scrambler on one service. It generates control words, interacts with a standard ECMG and insert ECM's. It also updates the PMT of the service to declare the ECM PID. The plugin cat modifies or creates the CAT to declare a new EMM stream. The plugin datainject implements a DVB SimulCrypt EMM/PDG inserter. It is used as a server by a standard EMMG to insert EMM's, just as if it was a MUX. Finally, the option -O on the last line specifies the output plugin and its arguments. Here, the output plugin is dektec which drives a Dektec modulator, a PCIe board in the computer.

For more details, refer to the TSDuck User's Guide.

Of course, such a complex command should be maintained and edited in a shell script. Unless of course we enjoy retyping it every time.

Because of its modular structure, TSDuck is better used with a flexible shell. On Windows systems, we may use PowerShell, Cygwin or MSys (DOS command line windows should now be reserved to polymorphic deviants only).

The PSI/SI table compiler

TSDuck provides many ways to manipulate PSI/SI and other types of MPEG tables. Tables can be extracted, examined, injected or modified on the fly using specific plugins for tsp.

Tables can be manipulated in several formats:

TSDuck utilities manipulate tables in any of these formats and converts between them. Specifically, the command tstabcomp is a table compiler. It takes XML source files as input, compiles the tables they contain and output binary section files. Thus, we can create any arbitrary table from scratch and build the corresponding binary sections to inject them in a transport stream. The command tstabcomp is also a table decompiler which can recreate the XML representation of tables from their binary sections.

More generally, all TSDuck commands can manipulate tables in binary or XML format equally. Tables can be captured from a live transport stream and directly saved in XML format. We can then edit the XML files to modify the tables and reinject them in any other transport stream.

Sample XML source file

The expected XML input format is fully documented in the TSDuck User's Guide. Briefly, an input XML file uses <tsduck> as root node. The root node contains any number of tables. The following sample XML file contains the definition for simple (and incomplete) PAT and PMT:

<?xml version="1.0" encoding="UTF-8"?>
<tsduck>

  <PAT version="8" transport_stream_id="0x0012" network_PID="0x0010">
    <service service_id="0x0001" program_map_PID="0x1234"/>
    <service service_id="0x0002" program_map_PID="0x0678"/>
  </PAT>

  <PMT version="4" service_id="0x0456" PCR_PID="0x1234">
    <CA_descriptor CA_system_id="0x0777" CA_PID="0x0251"/>
    <component elementary_PID="0x0567" stream_type="0x12">
      <CA_descriptor CA_system_id="0x4444" CA_PID="0x0252"/>
      <ISO_639_language_descriptor>
        <language code="fre" audio_type="0x45"/>
        <language code="deu" audio_type="0x78"/>
      </ISO_639_language_descriptor>
    </component>
  </PMT>

</tsduck>

Let's look at the various steps in a real example. We need to reproduce the case of a broadcast stream (satellite, terrestrial, whatever) with a specially crafted NIT. The plugin nit in tsp provides a convenient way to modify a NIT on the fly. But let's assume that the modifications we need are so specific that this plugin cannot do what we want.

First, we capture the original NIT from the live stream and save it in XML format:

tsp -I dvb --uhf 24 \
    -P tables --pid 16 --tid 0x40 --max 1 --xml nit.xml \
    -O drop

Now, we manually edit the XML file using any text editor and apply the specific modifications we need. Then, we can directly inject it in the stream:

tsp -I dvb --uhf 24 \
    -P inject nit.xml --pid 16 --replace --stuffing \
    -O dektec --uhf 24 --convolution 2/3 --guard 1/32