7. Linux RT Doorbell

The goal of the Linux application is to be the user-facing part of the RT Doorbell and provide a camera interface that sends JPEG-encoded camera frames to the Android app as soon as the user presses the Doorbell. To create the application, we are solving the following problems on Linux in C.

Read out JPEG-encoded frames from the camera
Send frames over UDP
Create an RT-optimized thread reacting to a GPIO falling edge to start capturing and sending frames over UDP

Follow the code examples and instructions for the specific utilization of the user space API and build separate applications that fulfill the requirements. Use the cross-compiler to build the programs for the target.

Sending UDP Packets

For sending UDP, take a look at the man page:

https://man7.org/linux/man-pages/man7/udp.7.html

UDP is a connectionless protocol and allows to send packets up to 2^16 Bytes.

Exercise

In C, build a program for the target which reads data from a jpg file and sends it via UDP to the host system. On the host, verify by using Wireshark, that the payload arrives as expected. In a further step, read and send the packet to the Android Application.

Here is a sample UDP server and client program: https://www.geeksforgeeks.org/udp-client-server-using-connect-c-implementation/

Reading out JPEG Frames from a UVC Camera

Use a USB camera with the client (or if not available on site, pass the host system's camera to the Linux guest on your notebook / desktop) and build an application that reads out frames via video4linux. On the target, make certain the uvcvideo driver is loaded (how do you load a driver at runtime?).

Source of information: https://www.kernel.org/doc/html/v5.15/userspace-api/media/v4l/capture.c.html

Reading out frames captured on a Linux device is utilizing Video for Linux v2 or short v4l2. Above Link describes the userspace interface. Build a program that reads out JPEG-encoded frames and writes their output to a file.

Putting it together

Build a new application that combines the functionality of the prior programs by constantly reading out camera frames and submitting them as packets via UDP.

GPIO

Take a look at the following documentation for the most recent API for GPIO in Linux:

Write a C application that utilizes the new GPIO interface for waiting until a pin is pulled to low (choose the same pin as in the user space lab). Here is some example code:

https://github.com/torvalds/linux/blob/master/tools/gpio/gpio-event-mon.c

Solution: Here is example code for listening on GPIO 21.

#include <linux/gpio.h>
#include <unistd.h>
#include <fcntl.h>
#include <string.h>
#include <errno.h>
#include <sys/ioctl.h>
#include <stdio.h>
#include <stdint.h>
#include <getopt.h>
#include <stdlib.h>
#include <sys/poll.h>

#define DEV_NAME "/dev/gpiochip0"
#define GPIO 21

int main(int argc, char *const *argv)
{

    struct gpioevent_request rq;

    int fd, ret;

    printf("gpio\n");

    memset(&rq, 0, sizeof(struct gpioevent_request));

    fd = open(DEV_NAME, O_RDONLY);
    if (fd < 0)
    {
        printf("Unabled to open %s: %s\n", DEV_NAME, strerror(errno));
        return -1;
    }

    rq.lineoffset = GPIO;
    rq.handleflags = GPIOHANDLE_REQUEST_INPUT | GPIOHANDLE_REQUEST_BIAS_PULL_UP;
    rq.eventflags = GPIOEVENT_EVENT_FALLING_EDGE;
    ret = ioctl(fd, GPIO_GET_LINEEVENT_IOCTL, &rq);
    close(fd);

    if (ret == -1)
    {
        printf("Unable to get line event from ioctl : %s\n", strerror(errno));
        return -1;
    }

    for (;;)
    {
        struct gpioevent_data event_data;

        printf("wating for falling edge\n");
        ret = read(rq.fd, &event_data, sizeof(event_data));

        if (ret != sizeof(event_data))
        {
            printf("Error while polling event from GPIO: %s\n", strerror(errno));
            break;
        }

        // Handle the event
        if (event_data.id == GPIOEVENT_EVENT_RISING_EDGE)
        {
            printf("Rising edge event detected on GPIO line %d\n", rq.lineoffset);
        }
        else if (event_data.id == GPIOEVENT_EVENT_FALLING_EDGE)
        {
            printf("Falling edge event detected on GPIO line %d\n", rq.lineoffset);
        }
    }

    close(rq.fd);

    return 0;
}

Note a couple of things:

Only Falling Edges will be detected.
The GPIO is configured as Pullup
The code is not very portable, instead of continuing to listen for a gpio event, we should have a method blocking until a falling edge has been detected.
When compiling this with the Cross Toolchain's gcc, the Kernel Headers of the toolchain are typically utilized. If the toolchain dates to an older Kernel, then it is necessary to supply the current Kernel's include files.

Here is a quick Makefile, that will help us compile gpio.c:

Makefile

CFLAGS=-I$(KERNEL_SOURCE)usr/include
CC=$(CROSS_COMPILE)gcc 

gpio: gpio.c
	$(CC) $(CFLAGS) gpio.c -o gpio

.PHONY install:
	scp gpio root@192.168.50.10:/root

Realtime Application

After patching the Kernel with PREEMPT_RT, we would now like to optimize our app to have as little latency as possible for starting to stream data. Therefore, we need to employ a couple of optimizations to achieve the best RT performance possible. Take a look at the following guide:

https://wiki.linuxfoundation.org/realtime/documentation/howto/applications/application_base

Build a simple application that has minimal latency and waits for the button to be pressed.

Here an example of the merged gpio code, which is now behaving optimized for RT:

/*
 * POSIX Real Time Example
 * using a single pthread as RT thread
 */

#include <limits.h>
#include <pthread.h>
#include <sched.h>
#include <stdio.h>
#include <sys/mman.h>
#include <linux/gpio.h>
#include <unistd.h>
#include <fcntl.h>
#include <string.h>
#include <errno.h>
#include <sys/ioctl.h>
#include <stdint.h>
#include <getopt.h>
#include <stdlib.h>
#include <sys/poll.h>

#define DEV_NAME "/dev/gpiochip0"
#define GPIO 21

void *thread_func(void *data)
{
    struct gpioevent_request rq;

    int fd, ret;

    printf("gpio\n");

    memset(&rq, 0, sizeof(struct gpioevent_request));

    fd = open(DEV_NAME, O_RDONLY);
    if (fd < 0)
    {
        printf("Unabled to open %s: %s\n", DEV_NAME, strerror(errno));
        return NULL;
    }

    rq.lineoffset = GPIO;
    rq.handleflags = GPIOHANDLE_REQUEST_INPUT | GPIOHANDLE_REQUEST_BIAS_PULL_UP;
    rq.eventflags = GPIOEVENT_EVENT_FALLING_EDGE;
    ret = ioctl(fd, GPIO_GET_LINEEVENT_IOCTL, &rq);
    close(fd);

    if (ret == -1)
    {
        printf("Unable to get line event from ioctl : %s\n", strerror(errno));
        return NULL;
    }

    for (;;)
    {
        struct gpioevent_data event_data;

        printf("wating for falling edge\n");
        ret = read(rq.fd, &event_data, sizeof(event_data));

        if (ret != sizeof(event_data))
        {
            printf("Error while polling event from GPIO: %s\n", strerror(errno));
            break;
        }

        // Handle the event
        if (event_data.id == GPIOEVENT_EVENT_RISING_EDGE)
        {
            printf("Rising edge event detected on GPIO line %d\n", rq.lineoffset);
        }
        else if (event_data.id == GPIOEVENT_EVENT_FALLING_EDGE)
        {
            printf("Falling edge event detected on GPIO line %d\n", rq.lineoffset);
        }
    }

    close(rq.fd);

    return NULL;
}

int main(int argc, char *argv[])
{
    struct sched_param param;
    pthread_attr_t attr;
    pthread_t thread;
    int ret;

    /* Lock memory */
    if (mlockall(MCL_CURRENT | MCL_FUTURE) == -1)
    {
        printf("mlockall failed: %m\n");
        exit(-2);
    }

    /* Initialize pthread attributes (default values) */
    ret = pthread_attr_init(&attr);
    if (ret)
    {
        printf("init pthread attributes failed\n");
        goto out;
    }

    /* Set a specific stack size  */
    ret = pthread_attr_setstacksize(&attr, PTHREAD_STACK_MIN);
    if (ret)
    {
        printf("pthread setstacksize failed\n");
        goto out;
    }

    /* Set scheduler policy and priority of pthread */
    ret = pthread_attr_setschedpolicy(&attr, SCHED_FIFO);
    if (ret)
    {
        printf("pthread setschedpolicy failed\n");
        goto out;
    }
    param.sched_priority = 80;
    ret = pthread_attr_setschedparam(&attr, &param);
    if (ret)
    {
        printf("pthread setschedparam failed\n");
        goto out;
    }
    /* Use scheduling parameters of attr */
    ret = pthread_attr_setinheritsched(&attr, PTHREAD_EXPLICIT_SCHED);
    if (ret)
    {
        printf("pthread setinheritsched failed\n");
        goto out;
    }

    /* Create a pthread with specified attributes */
    ret = pthread_create(&thread, &attr, thread_func, NULL);
    if (ret)
    {
        printf("create pthread failed\n");
        goto out;
    }

    /* Join the thread and wait until it is done */
    ret = pthread_join(thread, NULL);
    if (ret)
        printf("join pthread failed: %m\n");

out:
    return ret;
}

Of course it would be nicer, if the code was more modular (i.e. gpio.c would offer a function for detecting a falling edge).

Note that in order to compile rt.c it is necessary to also link the POSIX Thread Library with -lpthread.

Makefile

CFLAGS=-I$(KERNEL_SOURCE)usr/include
CC=$(CROSS_COMPILE)gcc 

gpio: gpio.c
	$(CC) $(CFLAGS) gpio.c -o gpio

rt: rt.c
	$(CC) $(CFLAGS) -lpthread rt.c -o rt

.PHONY install:
	scp gpio rt root@192.168.50.10:/root

Putting it all together

The Linux side of the RT Doorbell waits on the GPIO to be pulled to low and then starts sending JPEG data to the Android Application. After building a simple GPIO example in C, merge the program with the prior application to start sending JPEG data over UDP.

Build the full C application on the target side.

Here is a full implementiation of the Linux Part:

https://gitlab.fhnw.ch/ebssd/rt-doorbell

7. Linux RT Doorbell ​

Sending UDP Packets ​

Exercise ​

Reading out JPEG Frames from a UVC Camera ​

Putting it together ​

GPIO ​

Realtime Application ​

Putting it all together ​