Convert mp4 files to m2ts with ffmpeg

MontysEvilTwin

MontysEvilTwin

Well-Known Member
I unsuccessfully tried to convert an mp4 file to m2ts using ffmpeg with the following command:
Code: 
ffmpeg -i Test.mp4 -vcodec copy -acodec copy Test.m2ts
The process started OK, but it fell over quickly. Error message as follows:
Code: 
[mpegts @ 0x449be0] H.264 bitstream malformed, no startcode found, use the h264_
mp4toannexb bitstream filter                                                   
av_interleaved_write_frame(): Invalid data found when processing input
I was not trying to transcode; the HDR-FOX would probably struggle in this task. I just wanted to change the container to allow the creation of sidecar files. Would a later version of ffmpeg work? The custom firmware ffmpeg version is 0.10. I know that version 2.1.4 used to be available as a test package, but the link has gone now. The latest version of ffmpeg is 2.6.2: presumably this would have to be compiled to run on MIPS? @af123: please could you make version 2.1.4 available again so I can test it.
 
I think the command should be:
Code: 
ffmpeg -i Test.mp4 -vcodec copy -bsf h264_mp4toannexb Test.m2ts
Not sure if the m2ts output from ffmpeg would be compatible with the sidecar utility though. I seem to recall a problem with H264 video created by the ffmpeg codec because the video stream did not contain any H264 NAL units of type SEI . SEI timing units are an integral part of the broadcast stream, and are a requirement in order to create an nts file.
 
Last edited:
raydon said:
I think the command should be:
Code: 
ffmpeg -i Test.mp4 -vcodec copy -bsf h264_mp4toannexb Test.m2ts
Not sure if the m2ts output from ffmpeg would be compatible with the sidecar utility though. I seem to recall a problem with H264 video created by the ffmpeg codec because the video stream did not contain any H264 NAL units of type SEI . SEI timing units are an integral part of the broadcast stream, and are a requirement in order to create an nts file.
Click to expand...
Thanks, I'll try it out. I don't know much about video formats I just use trial and error. I was successful with Sidecar after converting non-native TS files to M2TS using ffmpeg (and renaming to TS) but, of course, the formats of these are very similar.
 
I tried the command in post 2 (I added '-acodec copy' to make sure the audio was copied too). This successfully created an m2ts file from the mp4 file. The file played fine on the HDR-FOX (when renamed '.TS') but, as indicated by Raydon, the sidecar package could not create an NTS file.
In case it is useful, I had a couple of mp4 files (h264 video, aac audio) I wanted to convert to TS with sidecars. I initially tried AV2HDR-T2 (Win7 PC), after remuxing to m2ts with TSmuxer GUI, but the file could not be processed. The Foxsat edition (I think it was version 3) of AV2HDR could process the m2ts file, so I took the TS file produced this way and reprocessed it ith AV2HDR-T2. This was partially successful: the file played but without audio.
I tried a few things to create files for processing by AV2HDR-T2, but avoided transcoding the video stream. The most successful involved creating an m2ts file from the mp4 with TSmuxer, then transcoding the audio to AC3 with AVIdemux. The resulting TS was fed into AV2HDR-TS. This worked, but the TS file did not play smoothly on the HDR-FOX. The playback issue was not related to the sidecars, the naked TS was similarly jerky on the HDR-FOX but played OK on a laptop and on an Android Tablet streamed over WiFi.
I have finally cracked it. An mp4 file (h264/ aac) was converted to m2ts with TSMuxer GUI. The file extension was renamed 'TS' and the sidecar files were created on the HDR-FOX using the sidecar package. The HMT file created had identical recording start and finish times so I manually edited this. Compared to a native TS file, fast forwarding and rewinding is less smooth at speeds >x8, but it still works and normal playback is fine.

EDIT. I repeated the sidecar generation from the command line. Here is the output:
Code: 
One# sidecar -hnpi Test                                                        
                                                                               
Test.ts                                                                        
Duration 0 seconds.                                                            
Video ID $1011, Type H264.                                                     
Audio ID $0000, Type MPEG.                                                     
Audio Desc ID $0000, Type not present.                                         
PMT ID $0100.                                                                  
Prog ID $0001.                                                                 
Channel Number 0.                                                              
Channel Name Freeview.                                                         
No event information available.                                                
Create HMT file.                                                               
HMT file creation complete.                                                    
Create NTS file.                                                               
Analyzing source video.                                                        
Indexing video frames [==================================================] 100%
Writing NTS file entries.                                                      
42623 NTS entries written.                                                     
NTS file creation complete.                                                    
                                                                               
Task completed in 2 mins 2 secs.
 
Last edited:
There was no error message as such. It seemed to be working; it said that it was processing video headers but just churned for ages. I then got a dialogue box saying "The page at 'IP address' says" with an OK button. Clicking OK, dismissed the message but the nts file contained no entries.
 
I think you need to try running sidecar on the command line to see if you're getting a similar crash problem to me. I would wager that you probably are.
 
I ran sidecar from the command line. Here is the output:
Code: 
One# sidecar -h -n -p -i Test                                                 
                                                                              
Test.ts                                                                       
Duration 1664 seconds.                                                        
Video ID $0100, Type H264.                                                    
Audio ID $0000, Type MPEG.                                                    
Audio Desc ID $0000, Type not present.                                        
PMT ID $1000.                                                                 
Prog ID $0001.                                                                
Channel Number 0.                                                             
Channel Name Freeview.                                                        
No event information available.                                               
Create HMT file.                                                              
HMT file creation complete.                                                   
Create NTS file.                                                              
Analyzing source video.                                                       
Indexing video frames [==================================================] 100%
Writing NTS file entries.
The process does not complete; only a 0B NTS file is created. Post #2 explains why it doesn't work. Ideally sidecar would return an error message stating that the file could not be processed and give the reason why before aborting the procedure, but I was using the package for a purpose it was not designed for.
 
I don't know whether this might throw some light on the issue that Monty raised.

Out of curiosity I wondered what would happen if I loaded an iPlayer .mp4 file into VideoReDo and saved it as H264 .m2ts, renamed it to .ts and ran the Ubuntu sidecar package that Raydon generously posted.

The sidecar package crashed and the full text output of sidecar (from start to crash) was as follows:

Code: 
/media/video-hd2/To Process/The World at War 02 (BBC iPlayer).ts
Duration 3094 seconds.
Video ID $0201, Type H264.
Audio ID $0000, Type MPEG.
Audio Desc ID $0000, Type not present.
PMT ID $0100.
Prog ID $0001.
Channel Number 2.
Channel Name BBC TWO.
No event information available.
Create HMT file.
HMT file creation complete.
Create NTS file.
Analyzing source video.
Indexing video frames [==================================================] 100%
Writing NTS file entries.
*** glibc detected *** sidecar: double free or corruption (fasttop): 0x093c6f40 ***
======= Backtrace: =========
/lib/i386-linux-gnu/libc.so.6(+0x74f82)[0xf75a9f82]
sidecar[0x804b369]
sidecar[0x804cbe3]
sidecar[0x804eb17]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf754e4d3]
======= Memory map: ========
08048000-08063000 r-xp 00000000 fc:00 1444270                            /usr/local/bin/sidecar
08063000-08069000 rw-p 0001b000 fc:00 1444270                            /usr/local/bin/sidecar
08069000-08186000 rw-p 00000000 00:00 0
093bc000-09ce3000 rw-p 00000000 00:00 0                                  [heap]
f7505000-f7521000 r-xp 00000000 fc:00 1308303                            /lib/i386-linux-gnu/libgcc_s.so.1
f7521000-f7522000 r--p 0001b000 fc:00 1308303                            /lib/i386-linux-gnu/libgcc_s.so.1
f7522000-f7523000 rw-p 0001c000 fc:00 1308303                            /lib/i386-linux-gnu/libgcc_s.so.1
f7533000-f7535000 rw-p 00000000 00:00 0
f7535000-f76d8000 r-xp 00000000 fc:00 1308732                            /lib/i386-linux-gnu/libc-2.15.so
f76d8000-f76da000 r--p 001a3000 fc:00 1308732                            /lib/i386-linux-gnu/libc-2.15.so
f76da000-f76db000 rw-p 001a5000 fc:00 1308732                            /lib/i386-linux-gnu/libc-2.15.so
f76db000-f76de000 rw-p 00000000 00:00 0
f76de000-f76e1000 r-xp 00000000 fc:00 1308729                            /lib/i386-linux-gnu/libdl-2.15.so
f76e1000-f76e2000 r--p 00002000 fc:00 1308729                            /lib/i386-linux-gnu/libdl-2.15.so
f76e2000-f76e3000 rw-p 00003000 fc:00 1308729                            /lib/i386-linux-gnu/libdl-2.15.so
f76ef000-f76f5000 rw-p 00000000 00:00 0
f76f5000-f76f6000 r-xp 00000000 00:00 0                                  [vdso]
f76f6000-f7716000 r-xp 00000000 fc:00 1308704                            /lib/i386-linux-gnu/ld-2.15.so
f7716000-f7717000 r--p 0001f000 fc:00 1308704                            /lib/i386-linux-gnu/ld-2.15.so
f7717000-f7718000 rw-p 00020000 fc:00 1308704                            /lib/i386-linux-gnu/ld-2.15.so
ff871000-ff892000 rw-p 00000000 00:00 0                                  [stack]
create-sidecars: line 2: 13219 Aborted                 (core dumped) sidecar -nhpi -c 2 -t "BBC TWO" "/media/video-hd2/To Process/The World at War 02 (BBC iPlayer)"

(The "create-sidecars: line 2" towards the end is because I was running the sidecar command in a script file "create-sidecars".)
 
johnb said:
I don't know whether this might throw some light on the issue that Monty raised.

Out of curiosity I wondered what would happen if I loaded an iPlayer .mp4 file into VideoReDo and saved it as H264 .m2ts, renamed it to .ts and ran the Ubuntu sidecar package that Raydon generously posted.

The sidecar package crashed and the full text output of sidecar (from start to crash) was as follows:

Code: 
/media/video-hd2/To Process/The World at War 02 (BBC iPlayer).ts
Duration 3094 seconds.
Video ID $0201, Type H264.
Audio ID $0000, Type MPEG.
Audio Desc ID $0000, Type not present.
PMT ID $0100.
Prog ID $0001.
Channel Number 2.
Channel Name BBC TWO.
No event information available.
Create HMT file.
HMT file creation complete.
Create NTS file.
Analyzing source video.
Indexing video frames [==================================================] 100%
Writing NTS file entries.
*** glibc detected *** sidecar: double free or corruption (fasttop): 0x093c6f40 ***
======= Backtrace: =========
/lib/i386-linux-gnu/libc.so.6(+0x74f82)[0xf75a9f82]
sidecar[0x804b369]
sidecar[0x804cbe3]
sidecar[0x804eb17]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf754e4d3]
======= Memory map: ========
08048000-08063000 r-xp 00000000 fc:00 1444270                            /usr/local/bin/sidecar
08063000-08069000 rw-p 0001b000 fc:00 1444270                            /usr/local/bin/sidecar
08069000-08186000 rw-p 00000000 00:00 0
093bc000-09ce3000 rw-p 00000000 00:00 0                                  [heap]
f7505000-f7521000 r-xp 00000000 fc:00 1308303                            /lib/i386-linux-gnu/libgcc_s.so.1
f7521000-f7522000 r--p 0001b000 fc:00 1308303                            /lib/i386-linux-gnu/libgcc_s.so.1
f7522000-f7523000 rw-p 0001c000 fc:00 1308303                            /lib/i386-linux-gnu/libgcc_s.so.1
f7533000-f7535000 rw-p 00000000 00:00 0
f7535000-f76d8000 r-xp 00000000 fc:00 1308732                            /lib/i386-linux-gnu/libc-2.15.so
f76d8000-f76da000 r--p 001a3000 fc:00 1308732                            /lib/i386-linux-gnu/libc-2.15.so
f76da000-f76db000 rw-p 001a5000 fc:00 1308732                            /lib/i386-linux-gnu/libc-2.15.so
f76db000-f76de000 rw-p 00000000 00:00 0
f76de000-f76e1000 r-xp 00000000 fc:00 1308729                            /lib/i386-linux-gnu/libdl-2.15.so
f76e1000-f76e2000 r--p 00002000 fc:00 1308729                            /lib/i386-linux-gnu/libdl-2.15.so
f76e2000-f76e3000 rw-p 00003000 fc:00 1308729                            /lib/i386-linux-gnu/libdl-2.15.so
f76ef000-f76f5000 rw-p 00000000 00:00 0
f76f5000-f76f6000 r-xp 00000000 00:00 0                                  [vdso]
f76f6000-f7716000 r-xp 00000000 fc:00 1308704                            /lib/i386-linux-gnu/ld-2.15.so
f7716000-f7717000 r--p 0001f000 fc:00 1308704                            /lib/i386-linux-gnu/ld-2.15.so
f7717000-f7718000 rw-p 00020000 fc:00 1308704                            /lib/i386-linux-gnu/ld-2.15.so
ff871000-ff892000 rw-p 00000000 00:00 0                                  [stack]
create-sidecars: line 2: 13219 Aborted                 (core dumped) sidecar -nhpi -c 2 -t "BBC TWO" "/media/video-hd2/To Process/The World at War 02 (BBC iPlayer)"

(The "create-sidecars: line 2" towards the end is because I was running the sidecar command in a script file "create-sidecars".)
Click to expand...
This is the exact same error that prpr has already reported. Thanks for your input.
 
I tested Sidecar v1.4 on a TS file created by converting an mp4 file (h264 video/ aac audio) to m2ts format with ffmpeg (the first example in post #4). Sidecar did not hang and did create a hmt file, though it was clearly much smaller than a regular nts file. As expected (see post #2), the TS file did not play correctly with the sidecar files in the same folder.
In the same post (#4) I reported that Sidecar could create valid sidecars for a TS file (actually a renamed m2ts file) created from a similar mp4 file, but processed using TSmuxer GUI (Windows). This works because SEI timing units are created when the streams are remuxed. The only issue was that the length of the recording was not set in the hmt file, requiring manual editing of the values stored at offsets 0x0284 and 0285. This is now fixed in the new version of Sidecar. However, I noticed that playback was video only: the audio track could not be heard. By comparing the v1.4 hmt file to the one created by v1.3, I found that the problem was caused by the variable at offset 0x0488. This is set to '00' by v1.4. If you change this to '01' the recording will play with audio.

Edit. To clarify, I am talking specifically about files converted from mp4. Sidecar is working fine with native TS files.
 
Last edited:
Determination of recording duration was improved in sidecar 1.4 which is why you didn't need to manually set it this time round.

The 2 bytes at offset 0x0488 is the Program ID in reverse byte order, so the PID of your source is 0x0001, as correctly listed by sidecar 1.3 in your post #4.
However, from the same listing, the source video appears to contain an MPEG audio track with an ID of 0x0000, which is very strange because the source audio is actually AAC. Also, the audio ID of zero is not a valid ID number. You can use the MediaInfo utility on your source to determine if the audio ID is correct.
I'm now only setting the PID value in the hmt file if the source is native, because using the PID/PMT from non-native sources has proven to be problematic in the past.
The PID is normally read from the .hmt by the T2 and used in conjunction with the stream list in the PMT (program map table) to determine the appropriate audio track and type. If the PID value is not set in the .hmt, then the T2 will instead determine the audio ID from the 2 byte value at offset 0x049C, and audio type from the single byte following at 0x04A0.

Can you run sidecar 1.4 on your source using the -i option and tell me if it still lists the audio as MPEG ?
Also, can I suggest that rather than changing the byte at 0x0488 to 1, you leave it alone and instead change the byte at offset 0x04A0 to 7 (AAC audio type) and see if the audio plays then. (You may also need to set the audio ID as well if MediaInfo shows it to be something other than 0x0000. Remember to reverse the byte order.).

I intend to to try and duplicate your method of creating a .ts from an .mp4 source because I need to determine why it is being incorrectly detected by sidecar as an MPEG stream, as I believe this is the real cause of your audio playback issue.

Edit: I have been able to duplicate this problem exactly and the audio ID is really 0x1100 (as reported by MediaInfo). However, patching the .hmt as I suggested still results in no audio playback. No need for you to do anything now Monty, just leave it with me.
 
Last edited:
With ffmpeg 4.1 uploaded to the repo earlier this year, a command like this seems to convert an iPlayer .mp4 into a .ts file suitable for sidecar:
Code: 
ffmpeg -hide_banner -nostats -i somefile.mp4 -c:v copy -c:a copy -mpegts_m2ts_mode true -mpegts_flags system_b -mpegts_service_type digital_tv somefile.ts

The ffmpeg mpegts muxer has these options:
Code: 
[code]
# ffmpeg -hide_banner -h muxer=mpegts
[code]
humaxhdr# ffmpeg -hide_banner -h muxer=mpegts
Muxer mpegts [MPEG-TS (MPEG-2 Transport Stream)]:
    Common extensions: ts,m2t,m2ts,mts.
    Mime type: video/MP2T.
    Default video codec: mpeg2video.
    Default audio codec: mp2.
MPEGTS muxer AVOptions:
  -mpegts_transport_stream_id <int>        E........ Set transport_stream_id field. (from 1 to 65535) (default 1)
  -mpegts_original_network_id <int>        E........ Set original_network_id field. (from 1 to 65535) (default 65281)
  -mpegts_service_id <int>        E........ Set service_id field. (from 1 to 65535) (default 1)
  -mpegts_service_type <int>        E........ Set service_type field. (from 1 to 255) (default digital_tv)
     digital_tv                   E........ Digital Television.
     digital_radio                E........ Digital Radio.
     teletext                     E........ Teletext.
     advanced_codec_digital_radio              E........ Advanced Codec Digital Radio.
     mpeg2_digital_hdtv              E........ MPEG2 Digital HDTV.
     advanced_codec_digital_sdtv              E........ Advanced Codec Digital SDTV.
     advanced_codec_digital_hdtv              E........ Advanced Codec Digital HDTV.
     hevc_digital_hdtv              E........ HEVC Digital Television Service.
  -mpegts_pmt_start_pid <int>        E........ Set the first pid of the PMT. (from 16 to 7936) (default 4096)
  -mpegts_start_pid  <int>        E........ Set the first pid. (from 16 to 3840) (default 256)
  -mpegts_m2ts_mode  <boolean>    E........ Enable m2ts mode. (default auto)
  -muxrate           <int>        E........ (from 0 to INT_MAX) (default 1)
  -pes_payload_size  <int>        E........ Minimum PES packet payload in bytes (from 0 to INT_MAX) (default 2930)
  -mpegts_flags      <flags>      E........ MPEG-TS muxing flags (default 0)
     resend_headers               E........ Reemit PAT/PMT before writing the next packet
     latm                         E........ Use LATM packetization for AAC
     pat_pmt_at_frames              E........ Reemit PAT and PMT at each video frame
     system_b                     E........ Conform to System B (DVB) instead of System A (ATSC)
     initial_discontinuity              E........ Mark initial packets as discontinuous
  -resend_headers    <int>        E........ Reemit PAT/PMT before writing the next packet (from 0 to INT_MAX) (default 0)
  -mpegts_copyts     <boolean>    E........ don't offset dts/pts (default auto)
  -tables_version    <int>        E........ set PAT, PMT and SDT version (from 0 to 31) (default 0)
  -omit_video_pes_length <boolean>    E........ Omit the PES packet length for video packets (default true)
  -pcr_period        <int>        E........ PCR retransmission time in milliseconds (from 0 to INT_MAX) (default 20)
  -pat_period        <double>     E........ PAT/PMT retransmission time limit in seconds (from 0 to INT_MAX) (default INT_MAX)
  -sdt_period        <double>     E........ SDT retransmission time limit in seconds (from 0 to INT_MAX) (default INT_MAX)
Apart from -mpegts_m2ts_mode, which of these ought to be set?
 
You must log in or register to reply here.
Back
Top