mirror of
https://github.com/octoleo/restic.git
synced 2024-12-05 03:13:18 +00:00
174 lines
8.1 KiB
Markdown
174 lines
8.1 KiB
Markdown
|
# Building `sys/unix`
|
||
|
|
||
|
The sys/unix package provides access to the raw system call interface of the
|
||
|
underlying operating system. See: https://godoc.org/golang.org/x/sys/unix
|
||
|
|
||
|
Porting Go to a new architecture/OS combination or adding syscalls, types, or
|
||
|
constants to an existing architecture/OS pair requires some manual effort;
|
||
|
however, there are tools that automate much of the process.
|
||
|
|
||
|
## Build Systems
|
||
|
|
||
|
There are currently two ways we generate the necessary files. We are currently
|
||
|
migrating the build system to use containers so the builds are reproducible.
|
||
|
This is being done on an OS-by-OS basis. Please update this documentation as
|
||
|
components of the build system change.
|
||
|
|
||
|
### Old Build System (currently for `GOOS != "Linux" || GOARCH == "sparc64"`)
|
||
|
|
||
|
The old build system generates the Go files based on the C header files
|
||
|
present on your system. This means that files
|
||
|
for a given GOOS/GOARCH pair must be generated on a system with that OS and
|
||
|
architecture. This also means that the generated code can differ from system
|
||
|
to system, based on differences in the header files.
|
||
|
|
||
|
To avoid this, if you are using the old build system, only generate the Go
|
||
|
files on an installation with unmodified header files. It is also important to
|
||
|
keep track of which version of the OS the files were generated from (ex.
|
||
|
Darwin 14 vs Darwin 15). This makes it easier to track the progress of changes
|
||
|
and have each OS upgrade correspond to a single change.
|
||
|
|
||
|
To build the files for your current OS and architecture, make sure GOOS and
|
||
|
GOARCH are set correctly and run `mkall.sh`. This will generate the files for
|
||
|
your specific system. Running `mkall.sh -n` shows the commands that will be run.
|
||
|
|
||
|
Requirements: bash, perl, go
|
||
|
|
||
|
### New Build System (currently for `GOOS == "Linux" && GOARCH != "sparc64"`)
|
||
|
|
||
|
The new build system uses a Docker container to generate the go files directly
|
||
|
from source checkouts of the kernel and various system libraries. This means
|
||
|
that on any platform that supports Docker, all the files using the new build
|
||
|
system can be generated at once, and generated files will not change based on
|
||
|
what the person running the scripts has installed on their computer.
|
||
|
|
||
|
The OS specific files for the new build system are located in the `${GOOS}`
|
||
|
directory, and the build is coordinated by the `${GOOS}/mkall.go` program. When
|
||
|
the kernel or system library updates, modify the Dockerfile at
|
||
|
`${GOOS}/Dockerfile` to checkout the new release of the source.
|
||
|
|
||
|
To build all the files under the new build system, you must be on an amd64/Linux
|
||
|
system and have your GOOS and GOARCH set accordingly. Running `mkall.sh` will
|
||
|
then generate all of the files for all of the GOOS/GOARCH pairs in the new build
|
||
|
system. Running `mkall.sh -n` shows the commands that will be run.
|
||
|
|
||
|
Requirements: bash, perl, go, docker
|
||
|
|
||
|
## Component files
|
||
|
|
||
|
This section describes the various files used in the code generation process.
|
||
|
It also contains instructions on how to modify these files to add a new
|
||
|
architecture/OS or to add additional syscalls, types, or constants. Note that
|
||
|
if you are using the new build system, the scripts cannot be called normally.
|
||
|
They must be called from within the docker container.
|
||
|
|
||
|
### asm files
|
||
|
|
||
|
The hand-written assembly file at `asm_${GOOS}_${GOARCH}.s` implements system
|
||
|
call dispatch. There are three entry points:
|
||
|
```
|
||
|
func Syscall(trap, a1, a2, a3 uintptr) (r1, r2, err uintptr)
|
||
|
func Syscall6(trap, a1, a2, a3, a4, a5, a6 uintptr) (r1, r2, err uintptr)
|
||
|
func RawSyscall(trap, a1, a2, a3 uintptr) (r1, r2, err uintptr)
|
||
|
```
|
||
|
The first and second are the standard ones; they differ only in how many
|
||
|
arguments can be passed to the kernel. The third is for low-level use by the
|
||
|
ForkExec wrapper. Unlike the first two, it does not call into the scheduler to
|
||
|
let it know that a system call is running.
|
||
|
|
||
|
When porting Go to an new architecture/OS, this file must be implemented for
|
||
|
each GOOS/GOARCH pair.
|
||
|
|
||
|
### mksysnum
|
||
|
|
||
|
Mksysnum is a script located at `${GOOS}/mksysnum.pl` (or `mksysnum_${GOOS}.pl`
|
||
|
for the old system). This script takes in a list of header files containing the
|
||
|
syscall number declarations and parses them to produce the corresponding list of
|
||
|
Go numeric constants. See `zsysnum_${GOOS}_${GOARCH}.go` for the generated
|
||
|
constants.
|
||
|
|
||
|
Adding new syscall numbers is mostly done by running the build on a sufficiently
|
||
|
new installation of the target OS (or updating the source checkouts for the
|
||
|
new build system). However, depending on the OS, you make need to update the
|
||
|
parsing in mksysnum.
|
||
|
|
||
|
### mksyscall.pl
|
||
|
|
||
|
The `syscall.go`, `syscall_${GOOS}.go`, `syscall_${GOOS}_${GOARCH}.go` are
|
||
|
hand-written Go files which implement system calls (for unix, the specific OS,
|
||
|
or the specific OS/Architecture pair respectively) that need special handling
|
||
|
and list `//sys` comments giving prototypes for ones that can be generated.
|
||
|
|
||
|
The mksyscall.pl script takes the `//sys` and `//sysnb` comments and converts
|
||
|
them into syscalls. This requires the name of the prototype in the comment to
|
||
|
match a syscall number in the `zsysnum_${GOOS}_${GOARCH}.go` file. The function
|
||
|
prototype can be exported (capitalized) or not.
|
||
|
|
||
|
Adding a new syscall often just requires adding a new `//sys` function prototype
|
||
|
with the desired arguments and a capitalized name so it is exported. However, if
|
||
|
you want the interface to the syscall to be different, often one will make an
|
||
|
unexported `//sys` prototype, an then write a custom wrapper in
|
||
|
`syscall_${GOOS}.go`.
|
||
|
|
||
|
### types files
|
||
|
|
||
|
For each OS, there is a hand-written Go file at `${GOOS}/types.go` (or
|
||
|
`types_${GOOS}.go` on the old system). This file includes standard C headers and
|
||
|
creates Go type aliases to the corresponding C types. The file is then fed
|
||
|
through godef to get the Go compatible definitions. Finally, the generated code
|
||
|
is fed though mkpost.go to format the code correctly and remove any hidden or
|
||
|
private identifiers. This cleaned-up code is written to
|
||
|
`ztypes_${GOOS}_${GOARCH}.go`.
|
||
|
|
||
|
The hardest part about preparing this file is figuring out which headers to
|
||
|
include and which symbols need to be `#define`d to get the actual data
|
||
|
structures that pass through to the kernel system calls. Some C libraries
|
||
|
preset alternate versions for binary compatibility and translate them on the
|
||
|
way in and out of system calls, but there is almost always a `#define` that can
|
||
|
get the real ones.
|
||
|
See `types_darwin.go` and `linux/types.go` for examples.
|
||
|
|
||
|
To add a new type, add in the necessary include statement at the top of the
|
||
|
file (if it is not already there) and add in a type alias line. Note that if
|
||
|
your type is significantly different on different architectures, you may need
|
||
|
some `#if/#elif` macros in your include statements.
|
||
|
|
||
|
### mkerrors.sh
|
||
|
|
||
|
This script is used to generate the system's various constants. This doesn't
|
||
|
just include the error numbers and error strings, but also the signal numbers
|
||
|
an a wide variety of miscellaneous constants. The constants come from the list
|
||
|
of include files in the `includes_${uname}` variable. A regex then picks out
|
||
|
the desired `#define` statements, and generates the corresponding Go constants.
|
||
|
The error numbers and strings are generated from `#include <errno.h>`, and the
|
||
|
signal numbers and strings are generated from `#include <signal.h>`. All of
|
||
|
these constants are written to `zerrors_${GOOS}_${GOARCH}.go` via a C program,
|
||
|
`_errors.c`, which prints out all the constants.
|
||
|
|
||
|
To add a constant, add the header that includes it to the appropriate variable.
|
||
|
Then, edit the regex (if necessary) to match the desired constant. Avoid making
|
||
|
the regex too broad to avoid matching unintended constants.
|
||
|
|
||
|
|
||
|
## Generated files
|
||
|
|
||
|
### `zerror_${GOOS}_${GOARCH}.go`
|
||
|
|
||
|
A file containing all of the system's generated error numbers, error strings,
|
||
|
signal numbers, and constants. Generated by `mkerrors.sh` (see above).
|
||
|
|
||
|
### `zsyscall_${GOOS}_${GOARCH}.go`
|
||
|
|
||
|
A file containing all the generated syscalls for a specific GOOS and GOARCH.
|
||
|
Generated by `mksyscall.pl` (see above).
|
||
|
|
||
|
### `zsysnum_${GOOS}_${GOARCH}.go`
|
||
|
|
||
|
A list of numeric constants for all the syscall number of the specific GOOS
|
||
|
and GOARCH. Generated by mksysnum (see above).
|
||
|
|
||
|
### `ztypes_${GOOS}_${GOARCH}.go`
|
||
|
|
||
|
A file containing Go types for passing into (or returning from) syscalls.
|
||
|
Generated by godefs and the types file (see above).
|