Hey,

I want to create a series of posts about me debugging stuff. I don’t know why, but it may be latently interesting with an educational message. Hopefully also to avoid other hackers wasting hours on the same issue I did.

FFMPEG

I’m currently working on a stream-based app at my job. I probably shouldn’t share more than is necessary but the necessary is enough - for reasons beyond the scope of this post, I need ffmpeg to take video data from standard input and publish them to a video server called Mediamtx.

Sounds simple enough and websites [ 1 ] and [ 2 ] provided partial solutions, which combined should be the solution but only if I didn’t make a grave mistake.

These are the commands in the articles:

python face-detection.py | ffmpeg  \
  -f rawvideo \
  -pixel_format bgr24 \
  -framerate 10 \
  -video_size 1280x720 \
  -i - foo.mp4 # ommiting HLS

ffmpeg -re -stream_loop -1 -i file.ts -c copy -f rtsp rtsp://localhost:8554/mystream

I first tested that the first command will work for my usecase - and success it did! To save the stream to a mp4 - one step away from my goal.

I checked the original commands and made educated guesses about the command options, looking up the ones I didn’t understand. Crucially, and the first step to disaster, ffmpeg --help full, which is supposed to show the entirety of all options of ffmpeg, didn’t work for me, for what I assumed were unrelated not-my-fault issues. Second issue ffmpeg --help does not explain the option called -re. It does however have an option called -report so I assumed that it is that, perhaps in a short version. More uncertainty was around the fact that I didn’t even know whether -re was one option or multiple one letter flags, as in other utilities, this would mean two options, -r and -e. The final step to disaster was my lack of knowledge of ffmpeg and me hand-waving any inconsistencies away.

so this is the command I made

cat stream.stdin | ffmpeg -f rawvideo -pixel_format bgr24 -framerate 10 -video_size 1920x1080 -i - -vcodec libx264 -f rtsp -rtsp_transport tcp rtsp://$destination_server/$destination_stream

An error occurred ❗ pipe broken. Alright. Fair enough, the standard input, stream.stdin part, of the command was the first suspect, being the second newest part of the program, causing extended time wasting. However after some thinking and testing I concluded there’s no reasonable way this is the issue.

I inspected the logs of both ffmpeg and Mediamtx further and this is where the disaster struck truly - Mediamtx reported ❗ timeout on connection!, ffmpeg reported ❗ broken pipe! at the exact same moment. This started a several hour wild goose chase of me trying to find out why do Mediamtx and ffmpeg seemingly fail to communicate properly. I increased timeouts, I tried different protocols, wondered forums and posts for hours, but nothing worked. So I came back to the original commands and I thought

hmm perhaps the -re, meaning report, might give me more information and it will make my command even more similar to the one given in the manual which has to be good too

I tried it and lo and behold the entire thing started to work, started up no problem. I was mildly confused, but at that point I realised that -re must be doing something else then -report. I tried again ffmpeg --help full, to no luck, and then i checked the man pages. And there I finally found it apparently -re changes the FPS of the input file!

That made me feel smart and dumb at the same time and a little annoyed. Not only was the issue not on the networking side, it wasn’t even on ffmpeg’s output side side. It was on the input side of ffmpeg, something that I deemed to be working fine once ffmpeg input to file.mp4 worked. And to add insult to injury, why did an incorrect FPS setting cause a ❗broken pipe!? Out of all things? If the error were corrupted input, bad encoding, I would understand…. but alas it is not meant to be. Why does setting the incorrect FPS even cause an error?

The lesson here is - don’t trust ffmpeg errors. The error reporting is useless at best and actually misleading at worst.

Also noteworthy - this is the second time ffmpeg’s fps screwed me over. The previous instance incorrect FPS setting manifested as ffmpeg getting stuck in the middle of streaming, for no obvious reason.

Also - the reason ffmpeg --help full didn’t work was because of a global alias on --help on my zsh shell, which I only investigated during the writing for this post. See [ 3 ]

TLDR: A confusing error from ffmpeg, my own hubris and my own defective environment has caused me to waste several hours. Oh well

[^ 1 ] https://gist.github.com/hiorws/92b65b55aae588a078a0e101aba266fe

[^ 2 ] https://github.com/bluenviron/mediamtx/tree/main#basic-usage

[^ 3 ] https://github.com/sharkdp/bat/issues/2771