Camera

Classes and functions for reading and writing camera streams.

A camera may be used to document participant responses on video or used by the experimenter to create movie stimuli or instructions.

Overview

Details

class psychopy.hardware.camera.Camera(device=0, mic=None, cameraLib='ffpyplayer', frameRate=None, frameSize=None, bufferSecs=4, win=None, name='cam', keepFrames=5, usageMode='video')

Class for displaying and recording video from a USB/PCI connected camera.

This class is capable of opening, recording, and saving camera video streams to disk. Camera stream reading/writing is done in a separate thread, allowing capture to occur in the background while the main thread is free to perform other tasks. This allows for capture to occur at higher frame rates than the display refresh rate. Audio recording is also supported if a microphone interface is provided, where recording will be synchronized with the video stream (as best as possible). Video and audio can be saved to disk either as a single file or as separate files.

GNU/Linux is supported only by the OpenCV backend (cameraLib=’opencv’).

Parameters:

• device (str or int) – Camera to open a stream with. If the ID is not valid, an error will be raised when open() is called. Value can be a string or number. String values are platform-dependent: a DirectShow URI or camera name on Windows, or a camera name/index on MacOS. Specifying a number (>=0) is a platform-independent means of selecting a camera. PsychoPy enumerates possible camera devices and makes them selectable without explicitly having the name of the cameras attached to the system. Use caution when specifying an integer, as the same index may not reference the same camera every time.

• mic (Microphone or None) – Microphone to record audio samples from during recording. The microphone input device must not be in use when record() is called. The audio track will be merged with the video upon calling save(). Make sure that Microphone.maxRecordingSize is specified to a reasonable value to prevent the audio track from being truncated. Specifying a microphone adds some latency to starting and stopping camera recording due to the added overhead involved with synchronizing the audio and video streams.

• frameRate (int or None) – Frame rate to record the camera stream at. If None, the camera’s default frame rate will be used.

• frameSize (tuple or None) – Size (width, height) of the camera stream frames to record. If None, the camera’s default frame size will be used.

• cameraLib (str) – Interface library (backend) to use for accessing the camera. May either be ffpyplayer or opencv. If None, the default library for the recommended by the PsychoPy developers will be used. Switching camera libraries could help resolve issues with camera compatibility. More camera libraries may be installed via extension packages.

• bufferSecs (float) – Size of the real-time camera stream buffer specified in seconds. This will tell the library to allocate a buffer that can hold enough frames to cover the specified number of seconds of video. This should be large enough to cover the time it takes to process frames in the main thread.

• win (Window or None) – Optional window associated with this camera. Some functionality may require an OpenGL context for presenting frames to the screen. If you are not planning to display the camera stream, this parameter can be safely ignored.

• name (str) – Label for the camera for logging purposes.

• keepFrames (int) – Number of frames to keep in memory for the camera stream. Calling getVideoFrames() will return the most recent keepFrames frames from the camera stream. If keepFrames is set to 0, no frames will be kept in memory and the camera stream will not be buffered. This is useful if the user desires to access raw frame data from the camera stream.

• latencyBias (float) – Latency bias to correct for asychrony between the camera and the microphone. This is the amount of time in seconds to add to the microphone recording start time to shift the audio track to match corresponding events in the video stream. This is needed for some cameras whose drivers do not accurately report timestamps for camera frames. Positive values will shift the audio track forward in time, and negative values will shift backwards.

• usageMode (str) – Usage mode hint for the camera aquisition. This with enable optimizations for specific applications that will improve performance and reduce memory usage. The default value is ‘video’, which is suitable for recording video streams with audio efficently. The ‘cv’ mode is for computer vision applications where frames from the camera stream are processed in real-time (e.g. object detection, tracking, etc.) and the video is not being saved to disk. Audio will not be recorded in this mode even if a microphone is provided.

Examples

Opening a camera stream and closing it:

camera = Camera(device=0)
camera.open()  # exception here on invalid camera
camera.close()

Recording 5 seconds of video and saving it to disk:

cam = Camera(0)
cam.open()
cam.record()  # starts recording

while cam.recordingTime < 5.0:  # record for 5 seconds
    if event.getKeys('q'):
        break
    cam.update()

cam.stop()  # stops recording
cam.save('myVideo.mp4')
cam.close()

Providing a microphone as follows enables audio recording:

mic = Microphone(0)
cam = Camera(0, mic=mic)

Overriding the default frame rate and size (if cameraLib supports it):

cam = Camera(0, frameRate=30, frameSize=(640, 480), cameraLib=u'opencv')

_assertCameraReady()

Assert that the camera is ready. Raises a CameraNotReadyError if the camera is not ready.

_assertMediaPlayer()

Assert that we have a media player instance open.

This will raise a RuntimeError if there is no player open. Use this function to ensure that a player is present before running subsequent code.

_closeMovieFileWriter()

Close the movie file writer.

This will close the movie file writer and free up any resources used by the writer. If the writer is not open, this will do nothing.

_closeMovieFileWriterFFPyPlayer()

Close the movie file writer using the FFPyPlayer library.

This will close the movie file writer and free up any resources used by the writer. If the writer is not open, this will do nothing.

_convertFrameToRGBFFPyPlayer(frame)

Convert a frame to RGB format.

This function converts a frame to RGB format. The frame is returned as a Numpy array. The resulting array will be in the correct format to upload to OpenGL as a texture.

Parameters:

frame (FFPyPlayer frame) – The frame to convert.

Returns:

The converted frame in RGB format.

Return type:

numpy.ndarray

_download()

Download video file to an online repository. Not implemented locally, needed for auto translate to JS.

_freeTextureBuffers()

Free any texture buffers used by the camera.

This is used to free up any texture buffers used by the camera. This is called when the camera is closed or when the window is closed.

_openMovieFileWriter(encoderLib=None, encoderOpts=None)

Open a movie file writer to save frames to disk.

This will open a movie file writer to save frames to disk. The frames will be saved to a temporary file and then merged with the audio track (if available) when save() is called.

Parameters:

• encoderLib (str or None) – Encoder library to use for saving the video. This can be either ‘ffpyplayer’ or ‘opencv’. If None, the same library that was used to open the camera stream. Default is None.

• encoderOpts (dict or None) – Options to pass to the encoder. This is a dictionary of options specific to the encoder library being used. See the documentation for ~psychopy.tools.movietools.MovieFileWriter for more details.

Returns:

Path to the temporary file that will be used to save the video. The file will be deleted when the movie file writer is closed or when save() is called.

Return type:

str

_openMovieFileWriterFFPyPlayer(filename, encoderOpts=None)

Open a movie file writer using the FFPyPlayer library.

Parameters:

• filename (str) – File to save the resulting video to, should include the extension.

• encoderOpts (dict or None) – Options to pass to the encoder. This is a dictionary of options specific to the encoder library being used. See the documentation for ~psychopy.tools.movietools.MovieFileWriter for more details.

_pixelTransfer()

Copy pixel data from video frame to texture.

This is called when a new frame is available. The pixel data is copied from the video frame to the texture store on the GPU.

_setupTextureBuffers()

Setup texture buffers for the camera.

This allocates OpenGL texture buffers for video frames to be written to which then can be rendered to the screen. This is only called if the camera is opened and a window is set.

_submitFrameToFile(frames, pts=None)

Submit a frame to the movie file writer thread.

This is used to submit frames to the movie file writer thread. It is called by the camera interface when a new frame is captured.

Parameters:

frames (MovieFrame) – Frame to submit to the movie file writer thread.

_submitFrameToFileFFPyPlayer(frames)

Submit a frame to the movie file writer thread using FFPyPlayer.

This is used to submit frames to the movie file writer thread. It is called by the camera interface when a new frame is captured.

Parameters:

frames (list of tuples) – Color data and presentation timestamps to submit to the movie file writer thread.

Returns:

Number of bytes written the the movie file.

Return type:

int

_upload()

Upload video file to an online repository. Not implemented locally, needed for auto translate to JS.

authorize()

Get permission to access the camera. Not implemented locally yet.

close()

Close the camera.

This will close the camera stream and free up any resources used by the device. If the camera is currently recording, this will stop the recording, but will not discard any frames. You may still call save() to save the frames to disk.

property colorTexture

OpenGL texture ID for the most recent video frame (int or None).

This is the OpenGL texture ID that can be used to render the most recent video frame to a window. If no window is set, this will be None.

property colorTextureSizeBytes

Size of the texture buffer used for rendering video frames (int or None).

This returns the size of the texture buffer in bytes used for rendering video frames. This is only valid if the camera is opened.

property device

Camera to use (str or None).

String specifying the name of the camera to open a stream with. This must be set prior to calling start(). If the name is not valid, an error will be raised when start() is called.

property frameCount

Total number of frames captured in the current recording (int).

This is the total number of frames captured since the last call to record(). This value is reset when record() is called again.

property frameInterval

Frame interval in seconds (float).

This is the time between frames in the video stream. This is computed from the frame rate of the video stream. If the frame rate is not set, this will return None.

property frameRate

Frame rate of the video stream (float or None).

Only valid after an open() and successive _enqueueFrame() call as metadata needs to be obtained from the stream. Returns None if not valid.

property frameSize

Size of the video frame obtained from recent metadata (float or None).

Only valid after an open() and successive _enqueueFrame() call as metadata needs to be obtained from the stream. Returns None if not valid.

getAudioTrack()

Get the audio track data.

Returns:

Audio track data from the microphone if available, or None if no microphone is set or no audio was recorded.

Return type:

AudioClip or None

static getCameraDescriptions(collapse=False)

Get a mapping or list of camera descriptions.

Camera descriptions are a compact way of representing camera settings and formats. Description strings can be used to specify which camera device and format to use with it to the Camera class.

Descriptions have the following format (example):

'[Live! Cam Sync 1080p] 160x120@30fps, mjpeg'

This shows a specific camera format for the ‘Live! Cam Sync 1080p’ webcam which supports 160x120 frame size at 30 frames per second. The last value is the codec or pixel format used to decode the stream. Different pixel formats and codecs vary in performance.

Parameters:

collapse (bool) – Return camera information as string descriptions instead of CameraInfo objects. This provides a more compact way of representing camera formats in a (reasonably) human-readable format.

Returns:

Mapping (dict) of camera descriptions, where keys are camera names (str) and values are a list of format description strings associated with the camera. If collapse=True, all descriptions will be returned in a single flat list. This might be more useful for specifying camera formats from a single GUI list control.

Return type:

dict or list

static getCameras(cameraLib='ffpyplayer')

Get information about installed cameras on this system.

Returns:

Mapping of camera information objects.

Return type:

dict

getLastClip()

File path to the last saved recording.

This value is only valid if a previous recording has been saved to disk (save() was called).

Returns:

Path to the file the most recent call to save() created. Returns None if no file is ready.

Return type:

str or None

getMetadata()

Get stream metadata.

Returns:

Metadata about the video stream, retrieved during the last frame update (_enqueueFrame call). If no metadata is available, returns None. This is useful for getting information about the video stream such as frame size, frame rate, pixel format, etc.

Return type:

MovieMetadata vor None

getRecentVideoFrame()

Get the most recent video frame from the camera.

Returns:

Most recent video frame. Returns None if no frame was available, or we timed out.

Return type:

VideoFrame or None

getVideoFrames()

Get the most recent frame from the stream (if available).

Returns:

List of recent video frames. This will return a list of frame images as numpy arrays, their presentation timestamp in the recording, and the absolute stream time in seconds. Frames will be converted to RGB format if they are not already. The number of frames returned will be limited by the keepFrames parameter set when creating the camera object. If no frames are available, an empty list will be returned.

Return type:

list of tuple

property hasMic

True if the camera has a microphone attached (bool).

This is True if the camera has a microphone attached and is ready to record audio. If the camera does not have a microphone, this will be False.

property interpolate

Whether the video texture should be filtered using linear or nearest neighbor interpolation (bool).

If True, the video texture will be filtered using linear interpolation. If False, the video texture will be filtered using nearest neighbor interpolation (pass-through). Default is True.

property isNotStarted

True if the stream may not have started yet (bool). This status is given before open() or after close() has been called on this object.

property isReady

True if the video and audio capture devices are in a ready state (bool).

When this is True, the audio and video streams are properly started.

property isRecording

True if the video is presently recording (bool).

property isStarted

True if the stream has started (bool). This status is given after open() has been called on this object.

property isStopped

True if the recording has stopped (bool). This does not mean that the stream has stopped, getVideoFrame() will still yield frames until close() is called.

property keepFrames

Number of frames to keep in memory for the camera stream (int).

property lastClip

File path to the last recording (str or None).

This value is only valid if a previous recording has been saved successfully (save() was called), otherwise it will be set to None.

property lastFrame

Most recent frame pulled from the camera (VideoFrame) since the last call of getVideoFrame.

property latencyBias

Latency bias in seconds (float).

This is the latency bias that is applied to the timestamps of the frames in the camera stream. This is useful for synchronizing the camera stream with other devices such as microphones or audio interfaces. The default value is 0.0, which means no latency bias is applied.

property metadata

Video metadata retrieved during the last frame update (MovieMetadata).

property mic

Microphone to record audio samples from during recording (Microphone or None).

If None, no audio will be recorded. Cannot be set after opening a camera stream.

open()

Open the camera stream and begin decoding frames (if available).

This function returns when the camera is ready to start getting frames.

Call record() to start recording frames to memory. Captured frames came be saved to disk using save().

poll()

Poll the camera for new frames.

Alias for update().

record(clearLastRecording=True, waitForStart=True)

Start recording frames.

This function will start recording frames and audio (if available). The value of lastFrame will be updated as new frames arrive and the frameCount will increase. You can access image data for the most recent frame to be captured using lastFrame.

If this is called before open() the camera stream will be opened automatically. This is not recommended as it may incur a longer than expected delay in the recording start time.

Warning

If a recording has been previously made without calling save() it will be discarded if record() is called again unless clearLastRecording=False.

Parameters:

• clearLastRecording (bool) – Clear the frame buffer before starting the recording. If True, the frame buffer will be cleared before starting the recording. If False, the frame buffer will be kept and new frames will be added to the buffer. Default is True. This is deprecated and will eventually be removed in a future version of PsychoPy. The recording is always cleared when record() is called, so this parameter is ignored.

• waitForStart (bool) – Capture video only when the camera and microphone are ready. This will result in a longer delay before the recording starts, but will ensure the microphone is actually recording valid samples. In some cases this will result in a delay of up to 1 second before the recording starts.

property recordingBytes

Current size of the recording in bytes (int).

property recordingTime

Current recording timestamp (float).

This returns the timestamp of the last frame captured in the recording.

This value increases monotonically from the last record() call. It will reset once stop() is called. This value is invalid outside record() and stop() calls.

save(filename, useThreads=True, mergeAudio=True, writerOpts=None)

Save the last recording to file.

This will write frames to filename acquired since the last call of record() and subsequent stop(). If record() is called again before save(), the previous recording will be deleted and lost.

This is a slow operation and will block for some time depending on the length of the video. This can be sped up by setting useThreads=True if supported.

Parameters:

• filename (str) – File to save the resulting video to, should include the extension.

• useThreads (bool) – Use threading where possible to speed up the saving process.

• mergeAudio (bool) – Merge the audio track from the microphone with the video into a single file if True. If False, the audio track will be saved to a separate file with the same name as filename, but with a .wav extension. This is useful if you want to process the audio track separately, or merge it with the video later on as the process is computationally expensive and memory consuming. Default is True.

• writerOpts (dict or None) – Options to pass to the movie writer. If None, default options will be used.

setTextureFilter(smooth=True)

Set whether the video texture should be filtered using linear or nearest neighbor interpolation.

Parameters:

smooth (bool) – If True, the video texture will be filtered using linear interpolation. If False, the video texture will be filtered using nearest neighbor interpolation (pass-through.) Default is True.

setWin(win)

Set the window to render the video frames to.

Parameters:

win (psychopy.visual.Window) – Window to render the video frames to. If None, no rendering will be done and the video frames will not be displayed.

start(waitForStart=True)

Start the camera stream.

This will start the camera stream and begin decoding frames. If the camera is already started, this will do nothing. Use record() to start recording frames to memory.

stop()

Stop recording frames and audio (if available).

property streamTime

Current stream time in seconds (float).

This is the current absolute time in seconds from the time the PC was booted. This is not the same as the recording time, which is the time since the recording started. This is useful for generating timestamps across multiple cameras or devices using the same time source.

update()

Acquire the newest data from the camera and audio streams.

This must be called periodically to ensure that stream buffers are flushed before they overflow to prevent data loss. Furthermore, calling this too infrequently may result also result in more frames needing to be processed at once, which may result in performance issues.

Returns:

Number of frames captured since the last call to this method. This will be 0 if no new frames were captured since the last call, indicating that the poll function is getting called too frequently or that the camera is not producing new frames (i.e. paused or closed). If -1 is returned, it indicates that the either or both the camera and microphone are not in a ready state albiet both interfaces are open. This can happen if update() is called very shortly after record().

Return type:

int

Examples

Capture camera frames in a loop:

while cam.recordingTime < 10.0:  # record for 10 seconds
    numFrames = cam.update()  # update the camera stream
    if numFrames > 0:
        frame = cam.getVideoFrame()  # get the most recent frame
        # do something with the frame, e.g. display it
    else:
        # return last frame or placeholder frame if nothing new

property win

Window to render the video frames to (psychopy.visual.Window or None).

If None, no rendering will be done and the video frames will not be displayed. If a window is set, the video frames will be rendered to the window using OpenGL textures.

class psychopy.hardware.camera.CameraInfo(index=-1, name='Null', frameSize=(-1, -1), frameRate=-1.0, pixelFormat='Unknown', codecFormat='Unknown', cameraLib='Null', cameraAPI='Null')

Information about a specific operating mode for a camera attached to the system.

Parameters:

• index (int) – Index of the camera. This is the enumeration for the camera which is used to identify and select it by the cameraLib. This value may differ between operating systems and the cameraLib being used.

• name (str) – Camera name retrieved by the OS. This may be a human-readable name (i.e. DirectShow on Windows), an index on MacOS or a path (e.g., /dev/video0 on Linux). If the cameraLib does not support this feature, then this value will be generated.

• frameSize (ArrayLike) – Resolution of the frame (w, h) in pixels.

• frameRate (ArrayLike) – Allowable framerate for this camera mode.

• pixelFormat (str) – Pixel format for the stream. If u’Null’, then codecFormat is being used to configure the camera.

• codecFormat (str) – Codec format for the stream. If u’Null’, then pixelFormat is being used to configure the camera. Usually this value is used for high-def stream formats.

• cameraLib (str) – Library used to access the camera. This can be either, ‘ffpyplayer’, ‘opencv’.

• cameraAPI (str) – API used to access the camera. This relates to the external interface being used by cameraLib to access the camera. This value can be: ‘AVFoundation’, ‘DirectShow’ or ‘Video4Linux2’.

property cameraAPI

Camera API in use to obtain this information (str).

property cameraLib

Camera library these settings are targeted towards (str).

property codecFormat

Codec format, may be used instead of pixelFormat for some configurations. Default is ‘’.

description()

Get a description as a string.

For all backends, this value is guaranteed to be valid after the camera has been opened. Some backends may be able to provide this information before the camera is opened.

Returns:

Description of the camera format as a human readable string.

Return type:

str

property frameRate

Frame rate (float) or range (ArrayLike).

Depends on the backend being used. If a range is provided, then the first value is the maximum and the second value is the minimum frame rate.

property frameSize

Resolution (w, h) in pixels (ArrayLike or None).

frameSizeAsFormattedString()

Get image size as as formatted string.

Returns:

Size formatted as ‘WxH’ (e.g. ‘480x320’).

Return type:

str

property index

Camera index (int). This is the enumerated index of this camera.

property name

Camera name (str). This is the camera name retrieved by the OS.

property pixelFormat

Video pixel format (str). An empty string indicates this field is not initialized.

Back to top