TSDuck users, we need your assistance to promote the project. Please read this.
TSDuck is an extensible toolkit for MPEG transport streams. It is used in digital television systems for test, monitoring, integration, debug, lab, demo.
In practice, TSDuck can be used for:
TSDuck is developed in C++ in a modular architecture. It is easy to extend it through plugins. In fact, the primary requirement for TSDuck architecture was to be able to implement any new feature within a couple of hours, which happened to be the case most of the time.
To be easily extended, TSDuck needed to be simple. TSDuck is a collection of command line tools and plugins. There is no sophisticated GUI. Each utility or plugin performs only one elementary feature but they can be combined in any order.
For C++ developers, TSDuck is also a large library to manipulate MPEG transport streams. Java and Python developers can use bindings in their languages for TSDuck high-level features.
In short, TSDuck shall be understood as a General-Purpose Toolbox for Digital TV Engineers and not as a specialized application for production operators.
And, by the way, TSDuck is Free and Open Source.
The most important TSDuck tool is tsp, the transport stream processor. It is a flexible transport stream processing framework. Running tsp is a combination of elementary processing operations using plugins.
In each invocation of tsp, there are:
This is illustrated in the diagram below:
The currently existing input and output plugins can work on TS files, UDP/IP (multicast or unicast), HTTP, HLS, SRT, DVB/ATSC/ISDB sources using cheap tuners, professional Dektec modulators and ASI devices, HiDes modulators.
This example describes a test bed or demo infrastructure for MPE injection and MPE extraction.
The network infrastructure is illustrated in the diagram below.
In network 1, a media server multicasts a transport stream on address 22.214.171.124, port 9000. We want to encapsulate this UDP multicast stream in an existing transport stream using MPE. This existing TS is a live satellite feed which is received using a DVB tuner.
We also change the multicast destination address for the UDP stream to 126.96.36.199, port 7000, in the MPE-encapsulated datagrams. There is no particular reason for this, we just illustrate the feasibility.
The resulting transport stream with embedded MPE is then broadcast. Here, the broadcast network is a Dektec modulator, followed by another computer using a DVB tuner.
This computer is connected to a second network. Another instance of tsp extracts the datagrams from the MPE stream and multicasts them on its network using the modified destination address.
The first tsp command adds two services in the existing transport stream. One service carries the IP/MAC Notification Table (INT). The other one carries the MPE stream. We create a fully DVB-conformant signalization for these two services.
The second tsp command extracts the MPE stream from the received TS and forwards all decapsulated UDP multicast datagrams on the local network.
This example is fully described in the TSDuck User's Guide with all details and corresponding commands.
Specific plugins such as merge and fork respectively combine and duplicate transport streams. They are designed to route transport streams from and to other applications. When the "other" application is another instance of tsp, we can create complex processing graphs.
This is illustrated in the diagram below.
In addition to tsp, the command tsswitch can be used to switch inputs between a set of transport streams using the same input and output plugins as tsp. Switching between inputs can be performed automatically at the end of each input or through a remote control application. The remote control protocol is extremely simple, based on text messages in UDP datagrams.
This is illustrated in the diagram below.
The remote control can be performed using simple command lines. The bash shell provides an easy way to redirect output to an UDP message. The following sample commands send UDP messages on port 4444 to system 127.0.0.1 (the local host). This is the easiest way to use the tsswitch remote control.
echo >/dev/udp/127.0.0.1/4444 2 echo >/dev/udp/127.0.0.1/4444 next
The following diagram illustrates a real-life example. To test a Conditional Access System and signalization, tsp is used to create a new transport stream from a live source. One real TS is received as input using a cheap USB or PCI DVB-T tuner. This TS and all its services are renamed, creating a "new TS" in the network. To test the CAS, tsp interacts with a real ECMG and a real EMMG using the standard DVB SimulCrypt protocols. One service, initially in the clear, is scrambled. tsp generates the control words, scrambles the content and inserts ECM's and EMM's. Finally, a set-top box receives all live transponders plus the new one we create.
Everything is done in one single tsp process. The command is the following. For clarity, the details such as tuning parameters, access criteria, IP addresses and ports, are represented as environment variables.
tsp -I dvb -u $UHF_INPUT \ -P tsrename -t 9 -a \ -P svrename direct8 -i 0x0901 -l 41 -n "Direct 8 Test" \ -P svrename bfmtv -i 0x0903 -l 42 -n "BFM TV Test" \ -P svrename 'i>tele' -i 0x0904 -l 43 -n "i>TELE Test" \ -P svrename virgin17 -i 0x0905 -l 44 -n "Virgin 17 Test" \ -P svrename gulli -i 0x0906 -l 45 -n "Gulli Test" \ -P svrename france4 -i 0x0907 -l 46 -n "France 4 Test" \ -P svrename 0x02FF -i 0x09FF \ -P scrambler GulliTest -e $ECMG -s $SUPER_CAS_ID \ -p $PMT_CADESC_PRIVATE -a $AC \ -b $ECM_BITRATE --pid $ECM_PID \ -P cat -c -a $CAS_ID/$EMM_PID/$CAT_CADESC_PRIVATE \ -P datainject -r -s $MUX_SERVER_PORT \ -b $EMM_MAX_BITRATE -p $EMM_PID \ -O dektec -u $UHF_OUTPUT --convolution 2/3 --guard 1/32
This command is complex but rational. The option -I on the first line specifies the input plugin and its arguments. Here, the input plugin is dvb which drives DVB tuners. Each option -P specifies a packet processor plugin and its options. The plugins tsrename and svrename respectively rename the transport stream and a service. The plugin scrambler is a DVB scrambler on one service. It generates control words, interacts with a standard ECMG and insert ECM's. It also updates the PMT of the service to declare the ECM PID. The plugin cat modifies or creates the CAT to declare a new EMM stream. The plugin datainject implements a DVB SimulCrypt EMM/PDG inserter. It is used as a server by a standard EMMG to insert EMM's, just as if it was a MUX. Finally, the option -O on the last line specifies the output plugin and its arguments. Here, the output plugin is dektec which drives a Dektec modulator, a PCIe board in the computer.
For more details, refer to the TSDuck User's Guide.
Of course, such a complex command should be maintained and edited in a shell script. Unless of course we enjoy retyping it every time.
Because of its modular structure, TSDuck is better used with a flexible shell. On Windows systems, we may use PowerShell, Cygwin or MSys (DOS command line windows should now be reserved to polymorphic deviants only).
TSDuck provides many ways to manipulate PSI/SI and other types of MPEG tables. Tables can be extracted, examined, injected or modified on the fly using specific plugins for tsp.
Tables can be manipulated in several formats:
TSDuck utilities manipulate tables in any of these formats and converts between them. Specifically, the command tstabcomp is a table compiler. It takes XML source files as input, compiles the tables they contain and output binary section files. Thus, we can create any arbitrary table from scratch and build the corresponding binary sections to inject them in a transport stream. The command tstabcomp is also a table decompiler which can recreate the XML representation of tables from their binary sections.
More generally, all TSDuck commands can manipulate tables in binary or XML format equally. Tables can be captured from a live transport stream and directly saved in XML format. We can then edit the XML files to modify the tables and reinject them in any other transport stream.
The expected XML input format is fully documented in the
TSDuck User's Guide.
Briefly, an input XML file uses
<?xml version="1.0" encoding="UTF-8"?> <tsduck> <PAT version="8" transport_stream_id="0x0012" network_PID="0x0010"> <service service_id="0x0001" program_map_PID="0x1234"/> <service service_id="0x0002" program_map_PID="0x0678"/> </PAT> <PMT version="4" service_id="0x0456" PCR_PID="0x1234"> <CA_descriptor CA_system_id="0x0777" CA_PID="0x0251"/> <component elementary_PID="0x0567" stream_type="0x12"> <CA_descriptor CA_system_id="0x4444" CA_PID="0x0252"/> <ISO_639_language_descriptor> <language code="fre" audio_type="0x45"/> <language code="deu" audio_type="0x78"/> </ISO_639_language_descriptor> </component> </PMT> </tsduck>
Let's look at the various steps in a real example. We need to reproduce the case of a broadcast stream (satellite, terrestrial, whatever) with a specially crafted NIT. The plugin nit in tsp provides a convenient way to modify a NIT on the fly. But let's assume that the modifications we need are so specific that this plugin cannot do what we want.
First, we capture the original NIT from the live stream and save it in XML format:
tsp -I dvb --uhf 24 \ -P tables --pid 16 --tid 0x40 --max 1 --xml nit.xml \ -O drop
Now, we manually edit the XML file using any text editor and apply the specific modifications we need. Then, we can directly inject it in the stream:
tsp -I dvb --uhf 24 \ -P inject nit.xml --pid 16 --replace --stuffing \ -O dektec --uhf 24 --convolution 2/3 --guard 1/32
TSDuck can also be used as an MPEG/DVB C++ library for your own applications. Most of the TSDuck code is located into a large shareable library. The utilities and plugins are usually only small wrappers around the TSDuck library. You may use the TSDuck library to develop new plugins or create completely independent applications.
The following diagram illustrates the TSDuck software architecture.
All C++ classes in the TSDuck library are documented using Doxygen.
There is a binary installer for the TSDuck development environment too. So, there is no need to rebuild TSDuck first. See more details here.
For Python and Java developers, TSDuck bindings exist to allow running transport stream processing pipelines from Python or Java applications. But since TSDuck is essentially a large C++ library, only a limited set of features is accessible from applications in other languages.
Here is a summary of all command line utilities and tsp plugins which are installed with TSDuck version 3.26-2257.
|tsanalyze||Analyze the structure of a transport stream|
|tsbitrate||Evaluate the bitrate of a transport stream|
|tscharset||Test tool for DVB and ARIB character sets|
|tscmp||Compare two transport stream files|
|tsdate||Extract the date and time (TDT/TOT) from a transport stream|
|tsdektec||Control Dektec devices|
|tsdump||Dump and format MPEG transport stream packets|
|tsecmg||Minimal generic DVB SimulCrypt-compliant ECMG|
|tsemmg||Minimal generic DVB SimulCrypt-compliant EMMG|
|tsfixcc||Fix continuity counters in a transport stream|
|tsftrunc||Truncate an MPEG transport stream file|
|tsgenecm||Generate one ECM using any DVB SimulCrypt compliant ECMG|
|tshides||List HiDes modulator devices|
|tslsdvb||List DVB tuner devices|
|tsp||MPEG transport stream processor using a chain of plugins|
|tspacketize||Packetize PSI/SI sections in a transport stream PID|
|tspcontrol||Send control commands to a running tsp|
|tspsi||Extract all standard PSI from an MPEG transport stream|
|tsresync||Resynchronize a non-standard or corrupted MPEG transport stream|
|tsscan||Scan a DTV network frequencies and services|
|tssmartcard||List or control smartcards|
|tsstuff||Add stuffing to a transport stream to reach a target bitrate|
|tsswitch||TS input source switch using remote control|
|tstabcomp||PSI/SI tables compiler|
|tstabdump||Dump PSI/SI tables, as saved by tstables|
|tstables||Collect PSI/SI tables from an MPEG transport stream|
|tsterinfo||Compute or convert DVB-Terrestrial information|
|tsversion||Check version, download and upgrade TSDuck|
|tsxml||Test tool for TSDuck XML manipulation|
|craft||Build specifically crafted input packets|
|dektec||Receive packets from a Dektec DVB-ASI or demodulator device|
|dvb||DVB receiver device input|
|file||Read packets from one or more files|
|fork||Fork a process and receive TS packets from its standard output|
|hls||Receive HTTP Live Streaming (HLS) media|
|http||Read a transport stream from an HTTP server|
|ip||Receive TS packets from UDP/IP, multicast or unicast|
|memory||Direct memory input from an application|
|null||Generate null packets|
|srt||Receive TS packets from Secure Reliable Transport (SRT)|
|dektec||Send packets to a Dektec DVB-ASI or modulator device|
|drop||Drop output packets|
|file||Write packets to a file|
|fork||Fork a process and send TS packets to its standard input|
|hides||Send packets to a HiDes modulator device|
|hls||Generate HTTP Live Streaming (HLS) media|
|ip||Send TS packets using UDP/IP, multicast or unicast|
|memory||Direct memory output to an application|
|play||Play output TS on any supported media player in the system|
|srt||Send TS packets using Secure Reliable Transport (SRT)|
|aes||Experimental AES scrambling of TS packets|
|analyze||Analyze the structure of a transport stream|
|bat||Perform various transformations on the BAT|
|bitrate_monitor||Monitor bitrate for TS or a given PID|
|boostpid||Boost the bitrate of a PID, stealing stuffing packets|
|cat||Perform various transformations on the CAT|
|clear||Extract clear (non scrambled) sequences of a transport stream|
|continuity||Check or fix continuity counters on TS packets|
|count||Count TS packets per PID|
|craft||Craft specific low-level transformations on packets|
|cutoff||Set labels on TS packets upon reception of UDP messages|
|datainject||DVB SimulCrypt data injector using EMMG/PDG <=> MUX protocol|
|decap||Decapsulate TS packets from a PID produced by encap plugin|
|descrambler||Generic DVB descrambler|
|duplicate||Duplicate PID's, reusing null packets|
|eit||Analyze EIT sections|
|encap||Encapsulate packets from several PID's into one single PID|
|file||Write packets to a file and pass them to next plugin|
|filter||Filter TS packets according to various conditions|
|fork||Fork a process and send TS packets to its standard input|
|history||Report a history of major events on the transport stream|
|inject||Inject tables and sections in a TS|
|limit||Limit the global bitrate by dropping packets|
|merge||Merge TS packets coming from the standard output of a command|
|mpe||Extract MPE (Multi-Protocol Encapsulation) datagrams|
|mpeinject||Inject an incoming UDP stream into MPE (Multi-Protocol Encapsulation)|
|mux||Insert TS packets in a transport stream|
|nit||Perform various transformations on the NIT|
|nitscan||Analyze the NIT and output a list of tuning information|
|pat||Perform various transformations on the PAT|
|pattern||Replace packet payload with a binary pattern on selected PID's|
|pcradjust||Adjust PCR's according to a constant bitrate|
|pcrbitrate||Permanently recompute bitrate based on PCR analysis|
|pcrextract||Extracts PCR, OPCR, PTS, DTS from TS packet for analysis|
|pcrverify||Verify PCR's from TS packets|
|pes||Analyze PES packets|
|pmt||Perform various transformations on the PMT|
|psi||Extract PSI Information|
|psimerge||Merge PSI/SI from mixed streams|
|reduce||Reduce the TS bitrate by removing stuffing packets|
|regulate||Regulate the TS packets flow based on PCR or bitrate|
|remap||Generic PID remapper|
|rmorphan||Remove orphan (unreferenced) PID's|
|rmsplice||Remove ads insertions from a program using SCTE 35 splice information|
|sdt||Perform various transformations on the SDT|
|sections||Remove or merge sections from various PID's|
|sifilter||Extract PID's containing the specified PSI/SI|
|skip||Skip leading TS packets of a stream|
|slice||Pass or drop packets based on packet numbers|
|spliceinject||Inject SCTE 35 splice commands in a transport stream|
|splicemonitor||Monitor SCTE 35 splice information|
|stats||Report various statistics on PID's and labels|
|stuffanalyze||Analyze the level of stuffing in tables|
|svremove||Remove a service|
|svrename||Rename a service, assign a new service name and/or new service id|
|t2mi||Extract T2-MI (DVB-T2 Modulator Interface) packets|
|tables||Collect PSI/SI Tables|
|teletext||Extract Teletext subtitles in SRT format|
|time||Schedule packets pass or drop, based on time|
|timeref||Update TDT and TOT with a new time reference|
|timeshift||Delay transmission by a fixed amount of packets|
|trigger||Trigger actions on selected TS packets|
|tsrename||Rename a transport stream|
|until||Copy packets until one of the specified conditions is met|
|zap||Zap on one service: Produce an SPTS containing only the specified service|
TSDuck is open source and released under the BSD license. It is developed in C++ and designed to be portable. TSDuck is built and tested for Windows, Linux and macOS. Please see the list of useful links in the side menu.
Pre-built binary packages are available for Windows, Fedora, Ubuntu, CentOS/RedHat (Intel) and Raspbian (Raspberry Pi). See the Download section in the side menu.
Rebuilding TSDuck, with or without its binary installers and packages, is documented here.
Support for specialized hardware, DVB tuners, Dektec devices, HiDes modulators, is available on Windows and Linux only.
Please report problems or ask questions using the issue tracker.
TSDuck is released under the terms of the license which is commonly referred to as 2-Clause BSD License or Simplified BSD License or FreeBSD License.
Copyright (c) 2005-2021, Thierry Lelégard
All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
This software is provided by the copyright holders and contributors "as is" and any express or implied warranties, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose are disclaimed. in no event shall the copyright owner or contributors be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to, procurement of substitute goods or services; loss of use, data, or profits; or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of this software, even if advised of the possibility of such damage.