Exploring Daemon Process Implementation in Go

Time： 2024-10-08 Column：Mobile & Frontend views：325

Implementing a daemon in Go can be challenging due to the language's runtime characteristics, but it is achievable through community-developed libraries and careful implementation.

In the world of backend development, the concept of a daemon is as old as Unix itself. A daemon is a long-running service program that operates in the background without being tied to any terminal. Although modern process management tools like systemd and supervisor make converting applications into daemons quite simple, we can even use the following command to run a program in the background:

nohup ./your_go_program &

However, in certain cases, it is still necessary for a program to natively transform into a daemon. For example, the mount subcommand of the distributed file system juicefs cli supports the -d option, allowing it to run as a daemon:

$juicefs mount -h
NAME:
   juicefs mount - Mount a volume

USAGE:
   juicefs mount [command options] META-URL MOUNTPOINT

...

OPTIONS:
   -d, --background  run in background (default: false)
   ...

This self-daemonizing capability can benefit many Go programs. In this article, we will explore how to transform Go applications into daemons.

1. Standard Daemonization Method

In W. Richard Stevens' classic book Advanced Programming in the UNIX Environment, the steps for daemonizing a program are detailed. The key steps are as follows:

Create a child process and terminate the parent process
Use the fork() system call to create a child process, and immediately terminate the parent process to ensure that the child process is not the session leader of the controlling terminal.
Create a new session
The child process calls setsid() to create a new session and become the session leader, thus detaching itself from the controlling terminal and process group.
Change the working directory
Use chdir("/") to change the current working directory to the root directory, preventing the daemon from holding any references to working directories, which could block file system unmounts.
Reset the file permission mask
Set the file mode creation mask to 0 using umask(0), allowing the daemon to set file permissions freely.
Close file descriptors
Close any open file descriptors inherited from the parent process, typically including standard input, output, and error.
Redirect standard input, output, and error
Reopen standard input, output, and error, redirecting them to /dev/null to prevent the daemon from accidentally outputting content to unintended places.

Note: The fork() system call can be a bit tricky to understand. It is used in UNIX/Linux systems to create a new process. The newly created process, called the child process, is a copy of the process that called fork() (i.e., the parent process). The child and parent processes share the same code, data, heap, and stack segments, but they are independent processes with different process IDs (PIDs). In the parent process, fork() returns the PID of the child (a positive integer), while in the child process, fork() returns 0. If fork() fails (for example, due to insufficient system resources), it returns -1, and errno is set to indicate the error cause.

Here is a C implementation of a daemonize function based on the classic steps from Advanced Programming in the UNIX Environment:

// daemonize/c/daemon.c

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <syslog.h>
#include <signal.h>

void daemonize()
{
    pid_t pid;

    // 1. Fork off the parent process 
    pid = fork();
    if (pid < 0) {
        exit(EXIT_FAILURE);
    }
    // If we got a good PID, then we can exit the parent process.
    if (pid > 0) {
        exit(EXIT_SUCCESS);
    }

    // 2. Create a new session to become session leader to lose controlling TTY
    if (setsid() < 0) {
        exit(EXIT_FAILURE);
    }

    // 3. Fork again to ensure the process won't allocate controlling TTY in future
    pid = fork();
    if (pid < 0) {
        exit(EXIT_FAILURE);
    }
    if (pid > 0) {
        exit(EXIT_SUCCESS);
    }

    // 4. Change the current working directory to root.
    if (chdir("/") < 0) {
        exit(EXIT_FAILURE);
    }

    // 5. Set the file mode creation mask to 0.
    umask(0);

    // 6. Close all open file descriptors.
    for (int x = sysconf(_SC_OPEN_MAX); x>=0; x--) {
        close(x);
    }

    // 7. Reopen stdin, stdout, stderr to /dev/null
    open("/dev/null", O_RDWR); // stdin
    dup(0);                    // stdout
    dup(0);                    // stderr

    // Optional: Log the daemon starting
    openlog("daemonized_process", LOG_PID, LOG_DAEMON);
    syslog(LOG_NOTICE, "Daemon started.");
    closelog();
}

int main() {
    daemonize();

    // Daemon process main loop
    while (1) {
        // Perform some background task...
        sleep(30); // Sleep for 30 seconds.
    }

    return EXIT_SUCCESS;
}

Note: The steps for setting system signal handlers are omitted here.

The daemonize function above completes the standard daemonization process and ensures that the program can run stably in the background without dependencies. After compiling and running the program, it runs in the background, and we can verify it using the ps command:

$ ./c-daemon-app 
$ ps -ef | grep c-daemon-app
root     28517     1  0 14:11 ?        00:00:00 ./c-daemon-app

We can see that the c-daemon-app has a parent process ID (PPID) of 1, which is the Linux init process. The C code above performs two fork() calls in the daemonization function. The reason for the two fork() calls is explained in my article Understanding Zombie and Daemon Processes, which I won’t repeat here.

Can Go implement a daemonization process similar to the steps above? Let’s explore that next.

2. Challenges of Implementing Daemon Processes in Go

The discussion on how to implement daemon processes in Go dates back to 2009, even before Go 1.0 was released. In the issue "runtime: support for daemonize"[5], the Go community and early contributors discussed the complexities of implementing native daemon processes in Go, mainly due to Go's runtime and thread management. When a process performs a fork operation, only the main thread is copied to the child process. If, before the fork, the Go program had multiple threads (which may have been spawned due to goroutines or garbage collection by the Go runtime), these non-forking threads (and their associated goroutines) would not be copied to the new child process. This can lead to unpredictable behavior in the child process, as some threads might leave data that can affect future execution.

Ideally, the Go runtime would provide a daemonize function, allowing daemonization before the multithreading starts. However, the Go team has not offered such a mechanism and instead suggests using third-party tools like systemd for managing Go processes as daemons.

Since Go does not provide a native solution, the Go community has found alternative approaches. Let's take a look at the solutions developed by the community.

3. Solutions from the Go Community

Despite the challenges, the Go community has developed several libraries to support daemon process implementation in Go. One popular solution is the github.com/sevlyar/go-daemon library.

The author of go-daemon cleverly solves the issue of not being able to directly use the fork system call in Go. The library simulates fork by using a simple but effective trick: it defines a special environment variable as a marker. When the program starts, it first checks whether this environment variable exists. If it does not, the parent process operations are executed, followed by launching a copy of the program with the environment variable set using os.StartProcess (essentially a fork-and-exec). If the environment variable exists, the child process operations are executed, and the main program logic continues. Below is the diagram provided by the author to explain this concept:

This method effectively simulates the behavior of fork while avoiding the problems related to threads and goroutines in the Go runtime. Here’s an example of how to implement a Go daemon using the go-daemon package:

// daemonize/go-daemon/main.go

package main

import (
	"log"
	"time"

	"github.com/sevlyar/go-daemon"
)

func main() {
	cntxt := &daemon.Context{
		PidFileName: "example.pid",
		PidFilePerm: 0644,
		LogFileName: "example.log",
		LogFilePerm: 0640,
		WorkDir:     "./",
		Umask:       027,
	}

	d, err := cntxt.Reborn()
	if err != nil {
		log.Fatal("Failed to run: ", err)
	}
	if d != nil {
		return
	}
	defer cntxt.Release()

	log.Print("Daemon started")

	// Daemon process logic
	for {
		// ... Perform some task ...
		time.Sleep(time.Second * 30)
	}
}

Once the program is running, you can check for the corresponding daemon process using ps:

$ make
go build -o go-daemon-app 
$ ./go-daemon-app 

$ ps -ef | grep go-daemon-app
  501  4025     1   0  9:20pm ??         0:00.01 ./go-daemon-app

Additionally, the program generates an example.pid file in the current directory (used to implement file locking) to prevent accidentally running the same go-daemon-app multiple times:

$ ./go-daemon-app
2024/09/26 21:21:28 Failed to run: daemon: Resource temporarily unavailable

Although native daemonization provides fine-grained control without requiring external dependencies, process management tools offer additional features such as auto-start on boot[6], automatic restart after abnormal termination, and logging. The Go team recommends using process management tools to handle Go daemons. The downside of these tools is that they require extra configuration (e.g., systemd) or setup (e.g., supervisor).

4. Conclusion

Implementing daemon processes in Go is challenging due to the characteristics of the Go runtime. However, it is achievable through community-developed libraries and careful implementation. As Go continues to evolve, we may see more native support for process management features. In the meantime, developers can choose between native daemonization, process management tools, or a hybrid approach based on their specific needs.

The source code for this article can be downloaded [here][7].

References: