Using an iPad as an external webcam with OBS on Linux - lmmx/devnotes GitHub Wiki

If you have:

  • an iPad (or other Apple AirPlay-enabled device)
  • a desktop computer or laptop running Linux Ubuntu 16 or higher (or a distro based on it)
  • 5GHz WiFi (i.e. not 2.4GHz), or a direct ethernet connection to such a network

Then you can use your iPad camera as an external webcam, and screen mirror the whole time to a PC that's screen recording, and afterwards overlay the recorded video from your iPad on top of the window (either the whole time, or just once to sync the two recordings).

After setting up uxplay, run it on your PC to open the connection socket.

Connect to this from your iPad (tap and hold "Screen Mirroring" in the pull down menu from the top right of the screen, then select "UxPlay").

Once connected, the X window server will just pass on the screen as a window. In the "Sources" list at the bottom of the OBS Studio interface, add a new source [by clicking the plus icon] of type "Window Capture (Xcomposite)", then select the uxplay window. Then you can treat the input just like a webcam, and crop it to how you want it (I suggest cropping out the camera button icons).

Now the window will be in your "scene" and you can click on it in the preview and drag it around to position, and scale its size.

  • You can also delete it: this might be useful if you're confident you only want to overlay it afterwards (I did so in ShotCut)

You can copy the recorded MOV file directly from the iPad over gvfs (when connected the device is mounted at run/user/1000/gvfs and it's slow but can be copied with cp, possibly you should use gio copy?)

Once you have the video on your PC, let's say it was IMG_1234.MOV, convert to MP4:

ffmpeg -i IMG_1234.MOV -vcodec copy -acodec copy IMG_1234.mp4

An optional next step is to improve the audio quality, open this file in audacity (it will automatically ignore the video and just read the audio track):

  • Run the noise reduction filter (first select a few seconds of noise and configure the noise profile then re-run noise reduction on the entire track)
  • Find the loudest parts (door slams, hand claps, etc) and select Amplify then choose a negative value to quieten them
  • After removing all of these peaks, increase the volume of the entire track by filtering with Amplify again. This time keep its automatic setting, and this will scale up the volume to the maximum (otherwise you'll have a quiet video).

(This comes built in when you use OBS with a video source)

Then imported the video file on top of the screen recording, and added 3 filters:

  • Opacity — to let me compare while resizing and repositioning the video against the one in the screen recording via the screen mirroring
    • This has a single setting, level, which is the % opacity
  • Size, Position, & Rotate — to scale down the recording to sit in the corner of the screen, on top of the screen mirrored window in the screen recording
    • Note that a little circular handle appears in the centre of the preview rectangle for the overlayed recorded video to help reposition it
  • Mask: Simple Shape
    • Operation should be set to Overwrite and Shape to Rectangle
    • Softness will feather the edge, turn this down to zero unless you want fuzzy blended edges on your video
    • Horizontal and Vertical move the centre of the mask (if not aligned with the video recording it will just show a black rectangle)
    • Width and Height will let you crop the mask (and therefore crop your video)

Lastly, find a particular facial expression and step through frame by frame to align the two videos at this frame (I found it helpful to split the videos with S and then compare these positions rather than memorising, then once you've calculated the gap that one video will need to be trimmed by to align the two recordings, undo the splits so you have complete video tracks again). Once you have the gap, just split (S) and delete (x) the surplus video segment and the consecutive segment will overwrite it