TSDuck is an extensible toolkit for MPEG/DVB transport streams. TSDuck is used in digital television systems for test, monitoring, integration, debug, lab, demo.
In practice, TSDuck is used for:
TSDuck is developed in C++ in a modular architecture. It is easy to extend it through plugins. In fact, the primary requirement for TSDuck architecture was to be able to implement any new feature within a couple of hours, which happened to be the case most of the time.
To be easily extended, TSDuck needed to be simple. TSDuck is a collection of command line tools and plugins. There is no sophisticated GUI. Each utility or plugin performs only one elementary feature but they can be combined in any order.
In short, TSDuck shall be understood as a general-purpose Toolbox for Digital TV Engineers and not as a specialized application for production operators.
The most important TSDuck tool is tsp, the transport stream processor. It is a flexible transport stream processing framework. Running tsp is a combination of elementary processing operations using plugins.
In each invocation of tsp, there are:
This is illustrated in the diagram below:
The currently existing input and output plugins can work on TS files, UDP/IP (multicast or unicast), DVB sources using cheap tuners, professional Dektec modulators and ASI devices, HiDes modulators.
The following diagram illustrates a real-life example. To test a Conditional Access System and signalization, tsp is used to create a new transport stream from a live source. One real TS is received as input using a cheap USB or PCI DVB-T tuner. This TS and all its services are renamed, creating a "new TS" in the network. To test the CAS, tsp interacts with a real ECMG and a real EMMG using the standard DVB SimulCrypt protocols. One service, initially in the clear, is scrambled. tsp generates the control words, scrambles the content and inserts ECM's and EMM's. Finally, a set-top box receives all live transponders plus the new one we create.
Everything is done in one single tsp process. The command is the following. For clarity, the details such as tuning parameters, access criteria, IP addresses and ports, are represented as environment variables.
tsp -I dvb -u $UHF_INPUT \ -P tsrename -t 9 -a \ -P svrename direct8 -i 0x0901 -l 41 -n "Direct 8 Test" \ -P svrename bfmtv -i 0x0903 -l 42 -n "BFM TV Test" \ -P svrename 'i>tele' -i 0x0904 -l 43 -n "i>TELE Test" \ -P svrename virgin17 -i 0x0905 -l 44 -n "Virgin 17 Test" \ -P svrename gulli -i 0x0906 -l 45 -n "Gulli Test" \ -P svrename france4 -i 0x0907 -l 46 -n "France 4 Test" \ -P svrename 0x02FF -i 0x09FF \ -P scrambler GulliTest -e $ECMG -s $SUPER_CAS_ID \ -p $PMT_CADESC_PRIVATE -a $AC \ -b $ECM_BITRATE --pid $ECM_PID \ -P cat -c -a $CAS_ID/$EMM_PID/$CAT_CADESC_PRIVATE \ -P datainject -r -s $MUX_SERVER_PORT \ -b $EMM_MAX_BITRATE -p $EMM_PID \ -O dektec -u $UHF_OUTPUT --convolution 2/3 --guard 1/32
This command is complex but rational. The option -I on the first line specifies the input plugin and its arguments. Here, the input plugin is dvb which drives DVB tuners. Each option -P specifies a packet processor plugin and its options. The plugins tsrename and svrename respectively rename the transport stream and a service. The plugin scrambler is a DVB scrambler on one service. It generates control words, interacts with a standard ECMG and insert ECM's. It also updates the PMT of the service to declare the ECM PID. The plugin cat modifies or creates the CAT to declare a new EMM stream. The plugin datainject implements a DVB SimulCrypt EMM/PDG inserter. It is used as a server by a standard EMMG to insert EMM's, just as if it was a MUX. Finally, the option -O on the last line specifies the output plugin and its arguments. Here, the output plugin is dektec which drives a Dektec modulator, a PCIe board in the computer.
For more details, refer to the TSDuck User's Guide.
Of course, such a complex command should be maintained and edited in a shell script. Unless of course you enjoy retyping it every time.
Because of its modular structure, TSDuck is better used with a flexible shell. On Windows systems, you may use PowerShell, Cygwin or MSys (DOS command line windows should now be reserved to polymorphic deviants only).
This example describes a test bed or demo infrastructure for MPE injection and MPE extraction. The network infrastructure is illustrated in the diagram below.
In network 1, a media server multicasts a transport stream on address 220.127.116.11, port 9000. We want to encapsulate this UDP multicast stream in an existing transport stream using MPE. This existing TS is a live satellite feed which is received using a DVB tuner.
We also change the multicast destination address for the UDP stream to 18.104.22.168, port 7000, in the MPE-encapsulated datagrams. There is no particular reason for this, we just illustrate the feasibility.
The resulting transport stream with embedded MPE is then broadcast. Here, the broadcast network is a Dektec modulator, followed by another computer using a DVB tuner.
This computer is connected to a second network. Another instance of tsp extracts the datagrams from the MPE stream and multicasts them on its network using the modified destination address.
The first tsp command adds two services in the existing transport stream. One service carries the IP/MAC Notification Table (INT). The other one carries the MPE stream. We create a fully DVB-conformant signalization for these two services.
The second tsp command extracts the MPE stream from the received TS and forwards all decapsulated UDP multicast datagrams on the local network.
This example is fully described in the TSDuck User's Guide with all details and corresponding commands.
Specific plugins such as merge and fork respectively combine and duplicate transport streams. They are designed to route transport streams from and to other applications. When the "other" application is another instance of tsp, we can create complex processing graphs.
This is illustrated in the diagram below.
TSDuck provides many ways to manipulate PSI/SI and other types of MPEG tables. Tables can be extracted, examined, injected or modified on the fly using specific plugins for tsp.
Tables can be manipulated in several formats:
TSDuck utilities manipulate tables in any of these formats and converts between them. Specifically, the command tstabcomp is a table compiler. It takes XML source files as input, compiles the tables they contain and output binary section files. Thus, you can create any arbitrary table from scratch and build the corresponding binary sections to inject them in a transport stream.
The expected XML input format is fully documented in the
TSDuck User's Guide.
Briefly, an input XML file uses
<?xml version="1.0" encoding="UTF-8"?> <tsduck> <PAT version="8" transport_stream_id="0x0012" network_PID="0x0010"> <service service_id="0x0001" program_map_PID="0x1234"/> <service service_id="0x0002" program_map_PID="0x0678"/> </PAT> <PMT version="4" service_id="0x0456" PCR_PID="0x1234"> <CA_descriptor CA_system_id="0x0777" CA_PID="0x0251"/> <component elementary_PID="0x0567" stream_type="0x12"> <CA_descriptor CA_system_id="0x4444" CA_PID="0x0252"/> <ISO_639_language_descriptor> <language code="fre" audio_type="0x45"/> <language code="deu" audio_type="0x78"/> </ISO_639_language_descriptor> </component> </PMT> </tsduck>
The command tstabcomp is also a table decompiler. If the input file is a binary section file, tstabcomp analyzes the tables and recreates the corresponding XML file.
One common need is to modify a binary table in a stream. Most of the time, a specialized plugin can do what you want on the fly. But if no predefined plugin exists for the precise modification you need, here is the most generic way to update a table:
Let's look at the various steps in a real example. We need to reproduce the case of a broadcast stream (satellite, terrestrial, whatever) with a specially crafted NIT. The plugin nit in tsp provides a convenient way to modify a NIT on the fly. But let's assume that the modifications we need are so specific that this plugin cannot do what we want.
First, we capture the original NIT from the live stream:
tsp -I dvb --uhf 24 \ -P tables --pid 16 --tid 0x40 --max 1 --bin nit.bin \ -O drop
Then, we decompile the binary table into an XML file. The input file being a
Now, manually edit the XML file using any text editor and apply the specific modifications we need. Then, recompile the XML file into a binary table.
Finally, we inject the recreated
tsp -I dvb --uhf 24 \ -P inject nit.bin --pid 16 --replace --stuffing \ -O dektec --uhf 24 --convolution 2/3 --guard 1/32
Now, we can plug a set-top box or TV set on the output of the modulator and test the result.
In fact, there is no need to explicitly decompile and recompile the table. Most TSDuck commands which manipulate tables and sections can use binary section files or XML file indifferently (see the TSDuck User's Guide for more details).
We can save the table directly in XML format:
tsp -I dvb --uhf 24 \ -P tables --pid 16 --tid 0x40 --max 1 --xml nit.xml \ -O drop
And, after modification of the XML file, we can directly inject it in the stream:
tsp -I dvb --uhf 24 \ -P inject nit.xml --pid 16 --replace --stuffing \ -O dektec --uhf 24 --convolution 2/3 --guard 1/32
TSDuck can also be used as an MPEG/DVB C++ library for your own applications. Most of the TSDuck code is located into a large shareable library. The utilities and plugins are usually only small wrappers around the TSDuck library. You may use the TSDuck library to develop new plugins or create completely independent applications.
All C++ classes in the TSDuck library are documented using Doxygen.
There is a binary installer for the TSDuck development environment too. So, there is no need to rebuild TSDuck first. See more details here.
Here is a summary of all command line utilities and tsp plugins which are installed with TSDuck version 3.15-903.
|tsanalyze||Analyze the structure of a transport stream|
|tsbitrate||Evaluate the bitrate of a transport stream|
|tscmp||Compare two transport stream files|
|tsdate||Extract the date and time (TDT/TOT) from a transport stream|
|tsdektec||Control Dektec devices|
|tsdump||Dump and format MPEG transport stream packets|
|tsecmg||Minimal generic DVB SimulCrypt-compliant ECMG|
|tsemmg||Minimal generic DVB SimulCrypt-compliant EMMG|
|tsfixcc||Fix continuity counters in a transport stream|
|tsftrunc||Truncate an MPEG transport stream file|
|tshides||List HiDes modulator devices|
|tslsdvb||List DVB tuner devices|
|tsp||MPEG transport stream processor using a chain of plugins|
|tspacketize||Packetize PSI/SI sections in a transport stream PID|
|tspsi||Extract all standard PSI from an MPEG transport stream|
|tsresync||Resynchronize a non-standard or corrupted MPEG transport stream|
|tsscan||Scan a DVB network|
|tssmartcard||List or control smartcards|
|tsstuff||Add stuffing to a transport stream to reach a target bitrate|
|tstabcomp||PSI/SI tables compiler|
|tstabdump||Dump PSI/SI tables, as saved by tstables|
|tstables||Collect PSI/SI tables from an MPEG transport stream|
|tsterinfo||Compute or convert DVB-Terrestrial information|
|tsversion||Check version, download and upgrade TSDuck|
|dektec||Receive packets from a Dektec DVB-ASI device|
|dvb||DVB receiver device input|
|file||Read packets from one or more files|
|http||Read a transport stream from an HTTP server|
|ip||Receive TS packets from UDP/IP, multicast or unicast|
|null||Generate null packets|
|dektec||Send packets to a Dektec DVB-ASI or modulator device|
|drop||Drop output packets|
|file||Write packets to a file|
|hides||Send packets to a HiDes modulator device|
|ip||Send TS packets using UDP/IP, multicast or unicast|
|play||Play output TS on any supported media player in the system|
|aes||Experimental AES scrambling of TS packets|
|analyze||Analyze the structure of a transport stream|
|bat||Perform various transformations on the BAT|
|bitrate_monitor||Monitor bitrate for a given pid|
|boostpid||Boost the bitrate of a PID, stealing stuffing packets.|
|cat||Perform various transformations on the CAT|
|clear||Extract clear (non scrambled) sequences of a transport stream|
|continuity||Check or fix continuity counters on TS packets|
|count||Count TS packets per PID|
|datainject||DVB SimulCrypt data injector using EMMG/PDG <=> MUX protocol|
|descrambler||Generic DVB descrambler.|
|eit||Analyze EIT sections|
|file||Write packets to a file and pass them to next plugin|
|filter||Filter TS packets according to various conditions|
|fork||Fork a process and send TS packets to its standard input|
|history||Report a history of major events on the transport stream|
|inject||Inject tables and sections in a TS|
|limit||Limit the global bitrate by dropping packets|
|merge||Merge TS packets coming from the standard output of a command|
|mpe||Extract MPE (Multi-Protocol Encapsulation) datagrams|
|mpeinject||Inject an incoming UDP stream into MPE (Multi-Protocol Encapsulation)|
|mux||Insert TS packets in a transport stream|
|nit||Perform various transformations on the NIT|
|nitscan||Analyze the NIT and output a list of tuning information|
|pat||Perform various transformations on the PAT|
|pattern||Replace packet payload with a binary pattern on selected PID's|
|pcrbitrate||Permanently recompute bitrate based on PCR analysis|
|pcrextract||Extracts PCR, OPCR, PTS, DTS from TS packet for analysis|
|pcrverify||Verify PCR's from TS packets|
|pes||Analyze PES packets|
|pmt||Perform various transformations on the PMT|
|psi||Extract PSI Information|
|reduce||Reduce the TS bitrate by removing stuffing packets|
|regulate||Regulate the TS packets flow based on PCR or bitrate|
|remap||Generic PID remapper.|
|rmorphan||Remove orphan (unreferenced) PID's|
|rmsplice||Remove ads insertions from a program using SCTE 35 splice information|
|sdt||Perform various transformations on the SDT|
|sections||Remove or merge sections from various PID's|
|sifilter||Extract PID's containing the specified PSI/SI|
|skip||Skip leading TS packets of a stream|
|slice||Pass or drop packets based on packet numbers|
|spliceinject||Inject SCTE 35 splice commands in a transport stream|
|stuffanalyze||Analyze the level of stuffing in tables|
|svremove||Remove a service|
|svrename||Rename a service, assign a new service name and/or new service id|
|t2mi||Extract T2-MI (DVB-T2 Modulator Interface) packets|
|tables||Collect PSI/SI Tables|
|teletext||Extract Teletext subtitles in SRT format|
|time||Schedule packets pass or drop, based on time|
|timeref||Update TDT and TOT with a new time reference|
|tsrename||Rename a transport stream|
|until||Copy packets until one of the specified conditions is met|
|zap||Zap on one service: Produce an SPTS containing only the specified service|
TSDuck is open source and released under the BSD license. It is developed in C++ and designed to be portable. TSDuck is built and tested for Windows, Linux and macOS. Please see the list of useful links in the side menu.
Pre-built binary packages are available for Windows, Fedora, Ubuntu (Intel) and Raspbian (Raspberry Pi). See the Download section in the side menu.
Rebuilding TSDuck, with or without its binary installers and packages, is documented here.
Support for specialized hardware, DVB tuners, Dektec devices, HiDes modulators, is available on Windows and Linux only.
Please report problems or ask questions using the issue tracker.
TSDuck is released under the terms of the license which is commonly referred to as 2-Clause BSD License or Simplified BSD License or FreeBSD License.
Copyright (c) 2005-2018, Thierry Lelégard
All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
This software is provided by the copyright holders and contributors "as is" and any express or implied warranties, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose are disclaimed. in no event shall the copyright owner or contributors be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to, procurement of substitute goods or services; loss of use, data, or profits; or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of this software, even if advised of the possibility of such damage.