jianguo
/
webcam-capture-driver-native
Template
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Add Mac driver

This commit is contained in:
Eduardo Ramos 2022-12-09 13:32:44 +01:00
parent 111542789e
commit bee4bb5e4a
9 changed files with 553 additions and 46 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

83
README.md
View File

@ -2,21 +2,23 @@
This is a native driver for [Webcam Capture](https://github.com/sarxos/webcam-capture) that is reliable, has very good performance, fast startup time and is able to correctly list the detailed capabilities of video devices such as resolutions and device IDs.
Currently it works on Windows only, with the `CaptureManagerDriver`, based on [CaptureManager-SDK](https://www.codeproject.com/Articles/1017223/CaptureManager-SDK-Capturing-Recording-and-Streami), which uses the MediaFoundation Windows API.
Currently it works on Windows and Mac.
For Windows, it uses the `CaptureManagerDriver`, based on [CaptureManager-SDK](https://www.codeproject.com/Articles/1017223/CaptureManager-SDK-Capturing-Recording-and-Streami), which uses the MediaFoundation Windows API.
For Mac, it uses `AVFDriver`, based on a [custom library](https://github.com/eduramiba/libvideocapture-avfoundation) that uses [AVFoundation](https://developer.apple.com/av-foundation/).
# How to use
1. Download this repository and run `mvn install`
2. Add `com.github.eduramiba:webcam-capture-driver-native:1.0.0-SNAPSHOT` dependency to your application.
3. Copy the DLLs of the `natives` folder for your system into the java library path.
4. Use the driver with `Webcam.setDriver(new CaptureManagerDriver())`
5. List the devices with `Webcam.getWebcams()` as normal and use the library in your preferred way. In JavaFX it's recommended to do it as in the example below.
3. Use the driver with `Webcam.setDriver(new NativeDriver())`
4. List the devices with `Webcam.getWebcams()` as normal and use the library in your preferred way. In JavaFX it's recommended to do it as in the example below.
# Simple example with JavaFX
```java
import com.github.eduramiba.webcamcapture.drivers.NativeDriver;
import com.github.eduramiba.webcamcapture.drivers.WebcamDeviceWithBufferOperations;
import com.github.eduramiba.webcamcapture.drivers.capturemanager.CaptureManagerDriver;
import com.github.sarxos.webcam.Webcam;
import com.github.sarxos.webcam.WebcamDevice;
import java.util.concurrent.Executors;
@ -32,62 +34,73 @@ import javafx.stage.Stage;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class TestCaptureManagerDriver extends Application {
public class TestDriver extends Application {
private static final Logger LOG = LoggerFactory.getLogger(TestDriver.class);
private static final Logger LOG = LoggerFactory.getLogger(TestCaptureManagerDriver.class);
public static final ScheduledExecutorService EXECUTOR = Executors.newScheduledThreadPool(4);
public static void main(String[] args) {
Webcam.setDriver(new CaptureManagerDriver());
Webcam.setDriver(new NativeDriver());
launch(args);
}
@Override
public void start(Stage stage) throws Exception {
ImageView imageView = new ImageView();
HBox root = new HBox();
final ImageView imageView = new ImageView();
final HBox root = new HBox();
root.getChildren().add(imageView);
Webcam.getWebcams().stream()
.findFirst()
.ifPresent((final Webcam camera) -> {
final WebcamDevice device = camera.getDevice();
LOG.info("Found camera: {}, device = {}", camera, device);
.findFirst()
.ifPresent((final Webcam camera) -> {
final WebcamDevice device = camera.getDevice();
LOG.info("Found camera: {}, device = {}", camera, device);
final int width = device.getResolution().width;
final int height = device.getResolution().height;
final WritableImage fxImage = new WritableImage(width, height);
Platform.runLater(() -> {
imageView.setImage(fxImage);
});
camera.open();
if (device instanceof WebcamDeviceWithBufferOperations) {
EXECUTOR.scheduleAtFixedRate(() -> {
((WebcamDeviceWithBufferOperations) device).updateFXIMage(fxImage);
}, 0, 16, TimeUnit.MILLISECONDS);
}
final int width = device.getResolution().width;
final int height = device.getResolution().height;
final WritableImage fxImage = new WritableImage(width, height);
Platform.runLater(() -> {
imageView.setImage(fxImage);
stage.setWidth(width);
stage.setHeight(height);
stage.centerOnScreen();
});
camera.getLock().disable();
camera.open();
if (device instanceof WebcamDeviceWithBufferOperations) {
EXECUTOR.scheduleAtFixedRate(() -> {
((WebcamDeviceWithBufferOperations) device).updateFXIMage(fxImage);
}, 0, 16, TimeUnit.MILLISECONDS);
}
});
stage.setOnCloseRequest(t -> {
Platform.exit();
System.exit(0);
});
// Create the Scene
Scene scene = new Scene(root);
// Add the scene to the Stage
final Scene scene = new Scene(root);
stage.setScene(scene);
// Set the title of the Stage
stage.setTitle("Displaying an Image");
// Display the Stage
stage.setTitle("Webcam example");
stage.show();
}
}
```
# Future work
* Publish this as a maven central artifact. At the moment you will need to build it yourself.
* Implement MacOS and Linux native driver that uses LibUVC.
* Implement Linux driver
# Notes
The source code in `natives` folder and `capturemanager` java package has been copied from [CaptureManager-SDK](https://www.codeproject.com/Articles/1017223/CaptureManager-SDK-Capturing-Recording-and-Streami) and slightly improved for this driver.
The source code in `natives` folder and `capturemanager` java package has been copied from [CaptureManager-SDK](https://www.codeproject.com/Articles/1017223/CaptureManager-SDK-Capturing-Recording-and-Streami) and slightly improved for this driver. This code is not idiomatic java and needs improvement.
The DLLs for Windows can just be copied along with your program.
The native dynamic libraries for Mac are on `src/main/resources` and loaded by JNA from inside the JAR.
Note that if you want to distribute a Mac app you will need to properly codesign the dylib files with entitlements, have an Info.plist, notarization...

21
pom.xml
View File

@ -21,6 +21,7 @@
<driver.webcam-capture.version>0.3.13-SNAPSHOT</driver.webcam-capture.version>
<driver.aalto-xml.version>1.3.2</driver.aalto-xml.version>
<driver.jna.version>5.12.1</driver.jna.version>
<driver.javafx.version>19</driver.javafx.version>
@ -55,6 +56,17 @@
<version>${driver.webcam-capture.version}</version>
</dependency>
<dependency>
<groupId>net.java.dev.jna</groupId>
<artifactId>jna</artifactId>
<version>${driver.jna.version}</version>
</dependency>
<dependency>
<groupId>net.java.dev.jna</groupId>
<artifactId>jna-platform</artifactId>
<version>${driver.jna.version}</version>
</dependency>
<dependency>
<groupId>com.fasterxml</groupId>
<artifactId>aalto-xml</artifactId>
@ -82,6 +94,15 @@
<artifactId>webcam-capture</artifactId>
</dependency>
<dependency>
<groupId>net.java.dev.jna</groupId>
<artifactId>jna</artifactId>
</dependency>
<dependency>
<groupId>net.java.dev.jna</groupId>
<artifactId>jna-platform</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml</groupId>
<artifactId>aalto-xml</artifactId>

28
src/main/java/com/github/eduramiba/webcamcapture/TestCaptureManagerDriver.java → src/main/java/com/github/eduramiba/webcamcapture/TestDriver.java
View File

@ -1,7 +1,7 @@
package com.github.eduramiba.webcamcapture;
import com.github.eduramiba.webcamcapture.drivers.NativeDriver;
import com.github.eduramiba.webcamcapture.drivers.WebcamDeviceWithBufferOperations;
import com.github.eduramiba.webcamcapture.drivers.capturemanager.CaptureManagerDriver;
import com.github.sarxos.webcam.Webcam;
import com.github.sarxos.webcam.WebcamDevice;
import java.util.concurrent.Executors;
@ -17,22 +17,22 @@ import javafx.stage.Stage;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class TestCaptureManagerDriver extends Application {
public class TestDriver extends Application {
private static final Logger LOG = LoggerFactory.getLogger(TestCaptureManagerDriver.class);
private static final Logger LOG = LoggerFactory.getLogger(TestDriver.class);
public static final ScheduledExecutorService EXECUTOR = Executors.newScheduledThreadPool(4);
public static void main(String[] args) {
Webcam.setDriver(new CaptureManagerDriver());
Webcam.setDriver(new NativeDriver());
launch(args);
}
@Override
public void start(Stage stage) throws Exception {
ImageView imageView = new ImageView();
HBox root = new HBox();
final ImageView imageView = new ImageView();
final HBox root = new HBox();
root.getChildren().add(imageView);
Webcam.getWebcams().stream()
@ -46,8 +46,12 @@ public class TestCaptureManagerDriver extends Application {
final WritableImage fxImage = new WritableImage(width, height);
Platform.runLater(() -> {
imageView.setImage(fxImage);
stage.setWidth(width);
stage.setHeight(height);
stage.centerOnScreen();
});
camera.getLock().disable();
camera.open();
if (device instanceof WebcamDeviceWithBufferOperations) {
EXECUTOR.scheduleAtFixedRate(() -> {
@ -56,13 +60,15 @@ public class TestCaptureManagerDriver extends Application {
}
});
stage.setOnCloseRequest(t -> {
Platform.exit();
System.exit(0);
});
// Create the Scene
Scene scene = new Scene(root);
// Add the scene to the Stage
final Scene scene = new Scene(root);
stage.setScene(scene);
// Set the title of the Stage
stage.setTitle("Displaying an Image");
// Display the Stage
stage.setTitle("Webcam example");
stage.show();
}
}

36
src/main/java/com/github/eduramiba/webcamcapture/drivers/NativeDriver.java Normal file
View File

@ -0,0 +1,36 @@
package com.github.eduramiba.webcamcapture.drivers;
import java.util.List;
import java.util.Locale;
import com.github.eduramiba.webcamcapture.drivers.avfoundation.driver.AVFDriver;
import com.github.eduramiba.webcamcapture.drivers.capturemanager.CaptureManagerDriver;
import com.github.sarxos.webcam.WebcamDevice;
import com.github.sarxos.webcam.WebcamDriver;
public class NativeDriver implements WebcamDriver {
private final WebcamDriver driver;
public NativeDriver() {
final String os = System.getProperty("os.name", "generic").toLowerCase(Locale.ENGLISH);
if ((os.indexOf("mac") >= 0) || (os.indexOf("darwin") >= 0)) {
this.driver = new AVFDriver();
} else if (os.indexOf("win") >= 0) {
this.driver = new CaptureManagerDriver();
} else {
throw new IllegalStateException("Unsupported OS = " + os);
}
}
@Override
public List<WebcamDevice> getDevices() {
return driver.getDevices();
}
@Override
public boolean isThreadSafe() {
return driver.isThreadSafe();
}
}

102
src/main/java/com/github/eduramiba/webcamcapture/drivers/avfoundation/driver/AVFDriver.java Normal file
View File

@ -0,0 +1,102 @@
package com.github.eduramiba.webcamcapture.drivers.avfoundation.driver;
import java.awt.*;
import java.nio.ByteBuffer;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Set;
import java.util.regex.Pattern;
import com.github.sarxos.webcam.WebcamDevice;
import com.github.sarxos.webcam.WebcamDriver;
import com.sun.jna.Native;
public class AVFDriver implements WebcamDriver {
private static final ByteBuffer buffer = ByteBuffer.allocateDirect(255);
@Override
public synchronized List<WebcamDevice> getDevices() {
final var lib = LibVideoCapture.INSTANCE;
final List<WebcamDevice> list = new ArrayList<>();
lib.vcavf_initialize();
final int count = lib.vcavf_devices_count();
if (count < 1) {
return list;
}
for (int devIndex = 0; devIndex < count; devIndex++) {
final String uniqueId = deviceUniqueId(devIndex);
final String name = deviceName(devIndex);
final int formatCount = lib.vcavf_get_device_formats_count(devIndex);
final Set<Dimension> resolutions = new LinkedHashSet<>();
for (int formatIndex = 0; formatIndex < formatCount; formatIndex++) {
final String format = deviceFormat(devIndex, formatIndex);
final Dimension resolution = formatToDimension(format);
if (resolution != null) {
resolutions.add(resolution);
}
}
final AVFVideoDevice device = new AVFVideoDevice(devIndex, uniqueId, name, resolutions);
if (device.isValid()) {
list.add(device);
}
}
return list;
}
@Override
public boolean isThreadSafe() {
return true;
}
private static String deviceUniqueId(final int deviceIndex) {
final var bufferP = Native.getDirectBufferPointer(buffer);
LibVideoCapture.INSTANCE.vcavf_get_device_unique_id(deviceIndex, bufferP, buffer.capacity());
return bufferP.getString(0, StandardCharsets.UTF_8.name());
}
private static String deviceModelId(final int deviceIndex) {
final var bufferP = Native.getDirectBufferPointer(buffer);
LibVideoCapture.INSTANCE.vcavf_get_device_model_id(deviceIndex, bufferP, buffer.capacity());
return bufferP.getString(0, StandardCharsets.UTF_8.name());
}
private static String deviceName(final int deviceIndex) {
final var bufferP = Native.getDirectBufferPointer(buffer);
LibVideoCapture.INSTANCE.vcavf_get_device_name(deviceIndex, bufferP, buffer.capacity());
return bufferP.getString(0, StandardCharsets.UTF_8.name());
}
private static String deviceFormat(final int deviceIndex, final int formatIndex) {
final var bufferP = Native.getDirectBufferPointer(buffer);
LibVideoCapture.INSTANCE.vcavf_get_device_format(deviceIndex, formatIndex, bufferP, buffer.capacity());
return bufferP.getString(0, StandardCharsets.UTF_8.name());
}
public static final Pattern RESOLUTION_PATTERN = Pattern.compile("[0-9]+x[0-9]+", Pattern.CASE_INSENSITIVE);
private static Dimension formatToDimension(final String format) {
final String[] parts = format.split(";");
if (parts.length > 0) {
final String resolution = parts[0].trim();
if (RESOLUTION_PATTERN.matcher(resolution).matches()) {
final String[] widthAndHeight = resolution.split("[Xx]");
return new Dimension(Integer.parseInt(widthAndHeight[0]), Integer.parseInt(widthAndHeight[1]));
}
}
return null;
}
}

280
src/main/java/com/github/eduramiba/webcamcapture/drivers/avfoundation/driver/AVFVideoDevice.java Normal file
View File

@ -0,0 +1,280 @@
package com.github.eduramiba.webcamcapture.drivers.avfoundation.driver;
import java.awt.*;
import java.awt.image.BufferedImage;
import java.awt.image.ComponentSampleModel;
import java.awt.image.DataBuffer;
import java.awt.image.DataBufferByte;
import java.awt.image.Raster;
import java.nio.ByteBuffer;
import java.util.Arrays;
import java.util.Collection;
import com.github.eduramiba.webcamcapture.drivers.WebcamDeviceWithBufferOperations;
import com.github.eduramiba.webcamcapture.drivers.WebcamDeviceWithId;
import com.github.sarxos.webcam.WebcamDevice;
import com.sun.jna.Native;
import javafx.scene.image.PixelFormat;
import javafx.scene.image.PixelWriter;
import javafx.scene.image.WritableImage;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import static com.github.eduramiba.webcamcapture.drivers.avfoundation.driver.LibVideoCapture.STATUS_AUTHORIZED;
public class AVFVideoDevice implements WebcamDevice, WebcamDevice.FPSSource, WebcamDevice.BufferAccess, WebcamDeviceWithId, WebcamDeviceWithBufferOperations {
private static final Logger LOG = LoggerFactory.getLogger(AVFVideoDevice.class);
private final int deviceIndex;
private final String id;
private final String name;
private final Dimension[] resolutions;
private Dimension resolution;
//State:
private boolean open = false;
private ByteBuffer imgBuffer = null;
private byte[] arrayByteBuffer = null;
private BufferedImage bufferedImage = null;
public AVFVideoDevice(final int deviceIndex, final String id, final String name, final Collection<Dimension> resolutions) {
this.deviceIndex = deviceIndex;
this.id = id;
this.name = name;
this.resolutions = resolutions != null ? resolutions.toArray(new Dimension[0]) : new Dimension[0];
this.resolution = bestResolution(this.resolutions);
}
public boolean isValid() {
return resolution != null && resolution.width > 0 && resolution.height > 0;
}
private Dimension bestResolution(final Dimension[] resolutions) {
Dimension best = null;
int bestPixels = 0;
for (Dimension dim : resolutions) {
int px = dim.width * dim.height;
if (px > bestPixels) {
best = dim;
bestPixels = px;
}
}
return best;
}
public int getDeviceIndex() {
return deviceIndex;
}
@Override
public String getId() {
return id;
}
@Override
public String getName() {
return name;
}
@Override
public Dimension[] getResolutions() {
return resolutions;
}
@Override
public Dimension getResolution() {
return resolution;
}
@Override
public void setResolution(Dimension resolution) {
if (isOpen()) {
return;
}
this.resolution = resolution;
}
@Override
public BufferedImage getImage() {
return getImage(imgBuffer);
}
@Override
public synchronized void open() {
if (isOpen()) {
return;
}
final var lib = LibVideoCapture.INSTANCE;
final int authStatus = lib.vcavf_has_videocapture_auth();
if (authStatus != STATUS_AUTHORIZED) {
LOG.warn("Capture auth status = {}", authStatus);
}
if (authStatus != STATUS_AUTHORIZED) {
lib.vcavf_ask_videocapture_auth();
}
final int width = resolution.width;
final int height = resolution.height;
final int startResult = lib.vcavf_start_capture(deviceIndex, width, height);
if (startResult < 0) {
LOG.warn("Capture start result for device {} = {}", id, startResult);
return;
}
final var bufferSizeBytes = width * height * 3;
this.open = true;
this.imgBuffer = ByteBuffer.allocateDirect(bufferSizeBytes);
this.bufferedImage = new BufferedImage(width, height, BufferedImage.TYPE_INT_BGR);
this.arrayByteBuffer = new byte[imgBuffer.capacity()];
LOG.info("Device {} opened successfully", id);
}
@Override
public synchronized void close() {
if (isOpen()) {
LibVideoCapture.INSTANCE.vcavf_stop_capture(deviceIndex);
open = false;
imgBuffer = null;
arrayByteBuffer = null;
bufferedImage = null;
}
}
@Override
public void dispose() {
}
@Override
public boolean isOpen() {
return open;
}
public static final int MAX_FPS = 30;
@Override
public double getFPS() {
//TODO: Use actual FPS declared by stream
return MAX_FPS;
}
@Override
public synchronized ByteBuffer getImageBytes() {
if (!isOpen()) {
return null;
}
updateBuffer();
return imgBuffer;
}
@Override
public synchronized void getImageBytes(final ByteBuffer target) {
if (!isOpen()) {
return;
}
if (target.remaining() < imgBuffer.capacity()) {
LOG.error("At least {} bytes needed but passed buffer has only {} remaining size", imgBuffer.capacity(), target.capacity());
return;
}
updateBuffer();
imgBuffer.rewind();
target.put(imgBuffer);
}
@Override
public String toString() {
return "AVFVideoDevice{" +
"deviceIndex=" + deviceIndex +
", id='" + id + '\'' +
", name='" + name + '\'' +
", resolutions=" + Arrays.toString(resolutions) +
'}';
}
@Override
public synchronized BufferedImage getImage(ByteBuffer byteBuffer) {
if (!isOpen()) {
return null;
}
updateBuffer();
updateBufferedImage();
return bufferedImage;
}
@Override
public synchronized boolean updateFXIMage(WritableImage writableImage) {
if (!isOpen()) {
return false;
}
updateBuffer();
return updateFXIMage(writableImage, imgBuffer);
}
public boolean updateFXIMage(final WritableImage writableImage, final ByteBuffer byteBuffer) {
if (!isOpen()) {
return false;
}
final int videoWidth = resolution.width;
final int videoHeight = resolution.height;
final PixelWriter pw = writableImage.getPixelWriter();
byteBuffer.mark();
byteBuffer.position(0);
pw.setPixels(
0, 0, videoWidth, videoHeight,
PixelFormat.getByteRgbInstance(), byteBuffer, 3 * videoWidth
);
return true;
}
private void updateBuffer() {
if (LibVideoCapture.INSTANCE.vcavf_has_new_frame(deviceIndex)) {
LibVideoCapture.INSTANCE.vcavf_grab_frame(
deviceIndex,
Native.getDirectBufferPointer(imgBuffer), imgBuffer.capacity());
}
}
private void updateBufferedImage() {
if (!isOpen()) {
return;
}
final int videoWidth = resolution.width;
final int videoHeight = resolution.height;
final ComponentSampleModel sampleModel = new ComponentSampleModel(
DataBuffer.TYPE_BYTE, videoWidth, videoHeight, 3, videoWidth * 3,
new int[]{0, 1, 2}
);
imgBuffer.mark();
imgBuffer.position(0);
imgBuffer.get(arrayByteBuffer, 0, imgBuffer.capacity());
imgBuffer.reset();
final DataBuffer dataBuffer = new DataBufferByte(arrayByteBuffer, arrayByteBuffer.length);
final Raster raster = Raster.createRaster(sampleModel, dataBuffer, null);
bufferedImage.setData(raster);
}
}

49
src/main/java/com/github/eduramiba/webcamcapture/drivers/avfoundation/driver/LibVideoCapture.java Normal file
View File

@ -0,0 +1,49 @@
package com.github.eduramiba.webcamcapture.drivers.avfoundation.driver;
import com.sun.jna.*;
public interface LibVideoCapture extends Library {
String JNA_LIBRARY_NAME = "videocapture";
NativeLibrary JNA_NATIVE_LIB = NativeLibrary.getInstance(LibVideoCapture.JNA_LIBRARY_NAME);
LibVideoCapture INSTANCE = Native.loadLibrary(LibVideoCapture.JNA_LIBRARY_NAME, LibVideoCapture.class);
public static final int RESULT_OK = (0);
public static final int ERROR_DEVICE_NOT_FOUND = (-1);
public static final int ERROR_FORMAT_NOT_FOUND = (-2);
public static final int ERROR_OPENING_DEVICE = (-3);
public static final int ERROR_SESSION_ALREADY_STARTED = (-4);
public static final int ERROR_SESSION_NOT_STARTED = (-5);
public static final int STATUS_AUTHORIZED = (0);
public static final int STATUS_NOT_DETERMINED = (-2);
public static final int STATUS_DENIED = (-1);
boolean vcavf_initialize();
int vcavf_has_videocapture_auth();
void vcavf_ask_videocapture_auth();
int vcavf_devices_count();
void vcavf_get_device_unique_id(int deviceIndex, Pointer pointer, int availableBytes);
void vcavf_get_device_model_id(int deviceIndex, Pointer pointer, int availableBytes);
void vcavf_get_device_name(int deviceIndex, Pointer pointer, int availableBytes);
int vcavf_get_device_formats_count(int deviceIndex);
void vcavf_get_device_format(int deviceIndex, int formatIndex, Pointer memory, int availableBytes);
int vcavf_start_capture(int deviceIndex, int width, int height);
int vcavf_stop_capture(int deviceIndex);
boolean vcavf_has_new_frame(int deviceIndex);
int vcavf_frame_width(int deviceIndex);
int vcavf_frame_height(int deviceIndex);
boolean vcavf_grab_frame(int deviceIndex, Pointer pointer, int availableBytes);
}

BIN
src/main/resources/darwin-aarch64/libvideocapture.dylib Executable file
View File

Binary file not shown.

BIN
src/main/resources/darwin-x86-64/libvideocapture.dylib Executable file
View File

Binary file not shown.