CBinding.jl
This package provides improvements for specifying and using C bindings in Julia. CBinding.jl has the goal of making it easier to correctly connect Julia to your C API and libraries. We have highlighted in our blog post many of the issues encountered when using C libraries from Julia, and those issues have fueled us to develop this package.
Usage
CBinding.jl provides some missing functionality and more precise specification capabilities than those provided by the builtin Julia facilities for interfacing C. All of the functionality and correctness of the CBinding.jl package has been compared to the behavior of GCC on x86_64 and AArch64 Linux distributions. Since many aspects of C are platform or compiler defined, the behavior of API's built for other platforms or compilers may not be matched by this package. Any help to test and develop against other setups is very much welcome!
C Aggregate and Array Types
CBinding.jl provides a statically-sized C array similar to the static array constructs found in some other Julia packages.
An array definition is obtained by using the@carray ElementType[SIZE]
syntax, which is provided to ease in the transcribing of C to Julia.
When used within the context of a @cunion
, @cstruct
, or @ctypedef
macro, ElementType[SIZE]
can be used directly to define arrays.
The union
and struct
aggregate types in C are described in very similar ways using CBinding.jl.
Both require the bit range of each aggregate field to be specified in order to support the different field packing approaches used in C.
Aggregate fields can be nested and anonymous aggregate fields can be used as well - two significant improvements over the usual Julian approach.
julia> using CBinding
julia> @cstruct MyFirstCStruct { # struct MyFirstCStruct {
i::Cint # int i;
} # };
julia> @ctypedef MySecondType @cstruct MySecondCStruct { # typedef struct MySecondCStruct {
i::Cint # int i;
j::Cint # int j;
@cunion { # union {
w::Cuchar[sizeof(Cint)÷sizeof(Cuchar)] # unsigned char w[sizeof(int)/sizeof(unsigned char)];
x::Cint # int x;
(y::_[4])::@cstruct { # struct {
c::Cuchar # unsigned char c;
} # } y[4];
z::MyFirstCStruct[1] # struct MyFirstCStruct z[1];
} # };
m::MyFirstCStruct # struct MyFirstCStruct m;
} # } MySecondType;
As you can see, type definition syntax closely mimics that of C, which you should find helpful when transcribing more complicated types or API's with numerous types. There are a few syntax differences to note though:
- a
@ctypedef
is specified with the type name before the definition rather than after (as is done in C) - likewise, an aggregate field is specified in the Julia
fieldName::FieldType
syntax rather than the C style ofFieldType fieldName
- in C a single line can specified multiple types (like
SomeType a, *b, c[4]
), but with our syntax these are expressed as a tuple ((a, b::Ptr{_}, c::_[4])::SomeType
) with the use of an underscore_
to mean "plug in type here" - Julia does not support forward declarations, so CBinding.jl uses some very dubious methods to simulate the capability which may cause grief and confusion for users when types don't appear as they should
The generic constructor provided by CBinding.jl allows you to create an aggregate with uninitialized values, zero-initialized values, or from existing an existing aggregate object.
It is also possible to use keyword arguments to specify values for particular fields within the aggregate as well.
Pass the zero
function to the constructor to create a zero-initialized object (an alternative is to pass the aggregate type to the zero
function).
The "undef" constructor is also defined and does nothing to initialize the memory region of the allocated object, so it is optimal to use in situations where an object will be fully initialized with particular values provided as keyword arguments.
julia> garbage = MySecondCStruct(undef) # struct MySecondCStruct garbage;
MySecondCStruct(i=-340722048, j=32586, w=UInt8[0x00, 0x00, 0x00, 0x00], x=0, y=<anonymous-struct>[(c=0x00), (c=0x00), (c=0x00), (c=0x00)], z=MyFirstCStruct[(i=0)], m=MyFirstCStruct(i=0))
julia> zeroed = MySecondCStruct(zero) # struct MySecondCStruct zeroed; memset(&zeroed, 0, sizeof(zeroed));
MySecondCStruct(i=0, j=0, w=UInt8[0x00, 0x00, 0x00, 0x00], x=0, y=<anonymous-struct>[(c=0x00), (c=0x00), (c=0x00), (c=0x00)], z=MyFirstCStruct[(i=0)], m=MyFirstCStruct(i=0))
Accessing the data fields within a C aggregate type works the way you would expect with one noteworthy detail.
Notice that when modifying fields within a union (e.g. zeroed.y[3].c = 0xff
) the change is also observed in the other fields in the union (zeroed.w
, zeroed.x
, and zeroed.y
).
julia> zeroed.i = 100
100
julia> zeroed
MySecondCStruct(i=100, j=0, w=UInt8[0x00, 0x00, 0x00, 0x00], x=0, y=<anonymous-struct>[(c=0x00), (c=0x00), (c=0x00), (c=0x00)], z=MyFirstCStruct[(i=0)], m=MyFirstCStruct(i=0))
julia> zeroed.y[3].c = 0xff
0xff
julia> zeroed
MySecondCStruct(i=100, j=0, w=UInt8[0x00, 0x00, 0xff, 0x00], x=16711680, y=<anonymous-struct>[(c=0x00), (c=0x00), (c=0xff), (c=0x00)], z=MyFirstCStruct[(i=16711680)], m=MyFirstCStruct(i=0))
When accessing a nested aggregate (or array) type, a Caccessor
object is used to maintain a reference to the enclosing object.
To get the aggregate itself that a Caccessor
is referring to you must use []
similar to Julia Ref
usage.
This will lead to some surprising results/behavior if you forget this detail.
The implemented Base.show
function will also cause the Caccessor
to appear as if you are working with the aggregate, so trust typeof
.
julia> typeof(zeroed.m)
Caccessor{MyFirstCStruct,MySecondCStruct,Val{12}}
julia> typeof(zeroed.m[])
MyFirstCStruct
julia> sizeof(zeroed.m)
8
julia> sizeof(zeroed.m[])
4
julia> zeroed.m
MyFirstCStruct(i=0)
julia> zeroed.m = MyFirstCStruct(zero, i = 42)
MyFirstCStruct(i=42)
julia> zeroed.m
MyFirstCStruct(i=42)
julia> zeroed.m[] = MyFirstCStruct(zero, i = 0)
MyFirstCStruct(i=0)
julia> zeroed.m
MyFirstCStruct(i=0)
C Field Alignment
By default, the fields in aggregates use native alignment to match the default alignment in C, but it is possible to denote packed aggregates using __packed__
, similar to using a __attribute__((packed))
attribute in C.
CBinding.jl also features the @calign
macro to describe additional alignment requirements when defining aggregate types.
julia> @cstruct MyUnalignedCStruct { # struct MyUnalignedCStruct {
c::Cchar # char c;
i::Cint # int i;
@cunion { # union {
f::Cfloat # float f;
d::Cdouble # double d;
} # };
} __packed__ # } __attribute__((packed));
MyUnalignedCStruct
julia> sizeof(MyUnalignedCStruct)
13
julia> @cstruct MyAlignedCStruct { # struct MyAlignedCStruct {
c::Cchar # char c;
i::Cint # int i;
@cunion { # union {
f::Cfloat # float f;
d::Cdouble # double d;
} # };
} # };
julia> sizeof(MyAlignedCStruct)
16
julia> @cstruct MyStrictlyAlignedCStruct { # struct MyStrictlyAlignedCStruct {
@calign 1 # align next field at 1 byte # alignas(1) char c;
c::Cchar # alignas(int) int i;
@calign sizeof(Cint) # align next field at 4 bytes # alignas(double) union {
i::Cint # alignas(float) float f;
@calign sizeof(Cdouble) # align largest nested field # alignas(double) double d;
@cunion { # };
@calign sizeof(Cfloat) # };
f::Cfloat
@calign sizeof(Cdouble)
d::Cdouble
}
}
MyStrictlyAlignedCStruct
julia> sizeof(MyStrictlyAlignedCStruct)
16
C Const-ness
One feature of C that doesn't necessarily port well into Julia is the const
modifier on types.
The aggregate types provided by CBinding.jl are mutable by default, but the Cconst
type can be used to create an immutable struct
form of the type.
This mechanism can be used in a way that is similar to the const
modifier in C.
julia> @cstruct ConstStruct { # struct ConstStruct {
i::Cint # int i;
j::Cconst{Cint} # int const j;
} # };
ConstStruct
julia> s = ConstStruct(zero, i = 1, j = 2)
ConstStruct(i=1, j=2)
julia> s.i = 3
3
julia> s
ConstStruct(i=3, j=2)
julia> s.j = 4
ERROR: Unable to change the value of a Cconst field
Stacktrace:
[1] error(::String) at ./error.jl:33
⁝
julia> c = Cconst(s)
Cconst(ConstStruct(i=3, j=2))
julia> c.i = 5
ERROR: Unable to change the value of a Cconst field
Stacktrace:
[1] error(::String) at ./error.jl:33
⁝
julia> s.i = 5
5
julia> s
ConstStruct(i=5, j=2)
julia> c # notice that `i` is not changed, `c` is a constant _copy_ of `s`
Cconst(ConstStruct(i=3, j=2))
Run time and memory usage can be improved by using Cconst
wrapped types as well.
If you do not wish to modify any fields in an aggregate, then it is recommended that you use the Cconst
form of the object.
C Enumerations
We also provide an implementation of C-style enumeration with a syntax very similar to that of C.
Enumerations may be defined by using the @cenum
macro in typedef or aggregate type macros and may be specified as either anonymous or named types.
The values of an enumeration must evaluate to integers, and can reference values defined earlier.
Usage of enumerations and values is generally promoted to integer arithmetic.
julia> @cenum MyNamedEnum { # enum MyNamedEnum {
VALUE_1, # VALUE_1,
VALUE_2, # VALUE_2,
VALUE_3, # VALUE_3
} # };
MyNamedEnum
julia> e = MyNamedEnum(VALUE_3)
MyNamedEnum(<VALUE_3>(0x00000002))
julia> e = MyNamedEnum(VALUE_1)
MyNamedEnum(<VALUE_1>(0x00000000))
julia> e | VALUE_3
2
julia> @cstruct EnumStruct { # struct EnumStruct {
e::@cenum { # enum {
X = 1<<0, # X = 1<<0,
Y = 1<<1, # Y = 1<<1,
Z = 1<<2, # Z = 1<<2
} # } e;
} # };
EnumStruct
julia> e = EnumStruct(zero)
EnumStruct(e=<anonymous-enum>(0x00000000))
julia> e.e = X|Y|Z
7
julia> e
EnumStruct(e=<anonymous-enum>(0x00000007))
CBinding.jl also allows you to apply an alignment strategy, such as __packed__
, to an enumeration definition.
An alignment strategy can be applied to both standalone enumeration types and enumerations nested within unions or structures.
julia> @cenum MyPackedNamedEnum { # enum MyPackedNamedEnum {
PACKED_VALUE_1, # PACKED_VALUE_1,
PACKED_VALUE_2, # PACKED_VALUE_2,
PACKED_VALUE_3, # PACKED_VALUE_3
} __packed__ # } __attribute__((packed));
MyPackedNamedEnum
julia> sizeof(MyNamedEnum)
4
julia> sizeof(MyPackedNamedEnum)
1
C Bit Fields
Specifying C bit fields is another feature provided by CBinding.jl.
Bit fields can be defined with (fieldName:FIELD_BITS)::FieldType
where FIELD_BITS
is an Integer number of bits and FieldType
is either Cint
or Cuint
.
julia> @cstruct BitfieldStruct { # struct BitfieldStruct {
(i:2)::Cint # int i:2;
(j:2)::Cuint # unsigned int j:2;
} # };
BitfieldStruct
julia> bf = BitfieldStruct(zero)
BitfieldStruct(i=0, j=0x00000000)
julia> sizeof(bf)
4
julia> for i in 1:5
bf.i = i ; @show bf
end
bf = BitfieldStruct(i=1, j=0x00000000)
bf = BitfieldStruct(i=-2, j=0x00000000)
bf = BitfieldStruct(i=-1, j=0x00000000)
bf = BitfieldStruct(i=0, j=0x00000000)
bf = BitfieldStruct(i=1, j=0x00000000)
julia> for i in 1:5
bf.j = i ; @show bf
end
bf = BitfieldStruct(i=1, j=0x00000001)
bf = BitfieldStruct(i=1, j=0x00000002)
bf = BitfieldStruct(i=1, j=0x00000003)
bf = BitfieldStruct(i=1, j=0x00000000)
bf = BitfieldStruct(i=1, j=0x00000001)
C Libraries
Interfacing C libraries is done through a Clibrary
object.
Once the library object is available, it can be used for obtaining global variables or functions directly.
This approach allows for multiple libraries to be loaded without causing symbol conflicts.
julia> lib = Clibrary() # dlopens the Julia process # void *lib = dlopen(NULL, RTLD_LAZY|RTLD_DEEPBIND|RTLD_LOCAL);
Clibrary(Ptr{Nothing} @0x000061eefd6a1000)
julia> lib2 = Clibrary("/path/to/library.so") # dlopens the library # void *lib2 = dlopen("/path/to/library.so", RTLD_LAZY|RTLD_DEEPBIND|RTLD_LOCAL);
Clibrary(Ptr{Nothing} @0x00006c1ce98c5000)
C Global Variables
A simple wrapper type, Cglobal
, is provided to obtain global variables from a library.
julia> val = Cglobal{Ptr{Cvoid}}(lib, :jl_nothing) # const void **val = dlsym(lib, "jl_nothing");
Cglobal{Ptr{Nothing}}(Ptr{Ptr{Nothing}} @0x00007fc384893bb8)
julia> val[] # dereference val
Ptr{Nothing} @0x00007fc3735ce008
The new @cextern
macro is the recommended method of binding global variables.
It is a more concise Julian representation that closely mirrors the C syntax.
This macro also includes the ability to use anonymous types in the definition of global variables.
julia> @cextern jl_base_module::Ptr{@cstruct jl_module_t} # extern struct _jl_module_t *jl_base_module;
Cglobal{Ptr{jl_module_t}}(Ptr{Ptr{jl_module_t}} @0x00007f84ce375230)
C Functions
This package adds the ability to specify function pointers in a type-safe way to Julia, similar to how you would in C.
You may specify a Cfunction
pointer directly, or use the constructor to load a symbol from a bound library.
The parametric types to Cfunction
are used to specify the return type and the tuple of argument types for the function referenced.
The additional type-safety will help you avoid many mishaps when calling C functions.
julia> CFuncType = Cfunction{Clong, Tuple{Ptr{Clong}}} # type of the function `long func(long *);`
Cfunction{Int64,Tuple{Ptr{Int64}},ConvT} where ConvT<:Cconvention
julia> CFuncPtrType = Ptr{CFuncType} # long (*func)(long *);
Ptr{Cfunction{Int64,Tuple{Ptr{Int64}},ConvT} where ConvT<:Cconvention}
julia> func = CFuncType(lib, :time) # long (*func)(long *) = dlsym(lib, "time");
Ptr{Cfunction{Int64,Tuple{Ptr{Int64}},ConvT} where ConvT<:Cconvention} @0x00007fff95de08c0
julia> @cstruct tm {
sec::Cint
min::Cint
hour::Cint
mday::Cint
mon::Cint
year::Cint
wday::Cint
yday::Cint
isdst::Cint
}
tm
julia> localtime = Cfunction{Ptr{tm}, Tuple{Ptr{Clong}}}(lib, :localtime) # struct tm *(*localtime)(long *) = dlsym(lib, "localtime");
Ptr{Cfunction{Ptr{tm},Tuple{Ptr{Int64}},ConvT} where ConvT<:Cconvention} @0x0000652bdb253fd0
CBinding.jl also makes a function pointer (Ptr{<:Cfunction}
) callable.
So, just as you would in C, you can simply call the function pointer to invoke it.
julia> func(C_NULL)
1560708358
julia> t = Ref(Clong(0))
Base.RefValue{Int64}(0)
julia> func(t)
1560708359
julia> t[] # dereference t
1560708359
julia> p = localtime(t)
Ptr{tm} @0x00007f4afa08b300
julia> unsafe_load(p) # dereference p
tm(sec=59, min=5, hour=14, mday=16, mon=5, year=119, wday=0, yday=166, isdst=1)
It is also possible to create type-safe function pointers to Julia functions for use in C code.
A closure is automatically created for the wrapped function and returned along with the C function pointer, so a reference to the closure (Base.CFunction
) must be kept to keep the function pointer valid.
One important thing to note is that the Julia function used is not (yet) guarded, so the argument and return types of the Julia function must match that of the Cfunction signature.
The new @ccallback
macro is now the recommended method of creating function pointers from Julia functions.
julia> (Cadd, add) = Cfunction{Cint, Tuple{Cint, Cint}}() do x::Cint, y::Cint
return Cint(x + y)
end
(Ptr{Cfunction{Int32,Tuple{Int32,Int32},Cconvention{:cdecl}}} @0x00007fc34e4dfa40, Base.CFunction(Ptr{Nothing} @0x00007fc34e4dfa40, getfield(Main, Symbol("##5#6"))(), Ptr{Nothing} @0x0000000000000000, Ptr{Nothing} @0x0000000000000000))
julia> Cadd(2, 3) # ccall the C function pointer, arguments are Base.cconvert-ed automatically
5
julia> add.f(Cint(2), Cint(3)) # directly call the Julia function
5
julia> Cadd = @ccallback function add(x::Cint, y::Cint)::Cint
return x + y
end
C Variadic Functions
Declaring a variadic function pointer can be done using a Vararg
argument type (which must be the last argument).
The variadic function calling capability provided with CBinding.jl is not limited in the ways that native Julia ccall usage is.
This enables Julia the ability to perform real-world variadic function usage as demonstrated with an example of binding to printf
and then calling it below.
julia> func = Cfunction{Cint, Tuple{Cstring, Vararg}}(lib, :printf) # int (*func)(char *, ...) = dlsym(lib, "printf");
Ptr{Cfunction{Int32,Tuple{Cstring,Vararg{Any,N} where N},ConvT} where ConvT<:Cconvention} @0x000061eefc388930
julia> func("%s i%c %ld great demo of CBinding.jl v%3.1lf%c\n", "this", 's', 1, 0.1, '!')
this is 1 great demo of CBinding.jl v0.1!
42
Binding Julia with a C library
The new @cbindings
and @cextern
macros provide the recommended method of binding C functions with Julia.
Its intended use is for creating optimized function bindings rather than loading function pointers.
It provides a more concise Julian representation that closely mirrors the C syntax.
The @cextern
macro also includes the ability to use anonymous types in the definition of function.
julia> @cextern time(ptr::Ptr{Clong})::Clong
time (generic function with 1 method)
julia> @cextern localtime(ptr::Ptr{Clong})::Ptr{tm}
localtime (generic function with 1 method)
julia> @cextern printf(format::Cstring, vals...)::Cint
printf (generic function with 1 method)
julia> time(t)
1560708359
julia> unsafe_load(localtime(t))
tm(sec=59, min=5, hour=14, mday=16, mon=5, year=119, wday=0, yday=166, isdst=1)
julia> printf("%s i%c %ld great demo of CBinding.jl v%3.1lf%c\n", "this", 's', 1, 0.1, '!')
this is 1 great demo of CBinding.jl v0.1!
42
The @cbindings
macro can be used to cleanly group and target bindings to a particular library.
Any @ctypedef
, @cstruct
, @cextern
, etc. macro usage can be placed within a @cbindings
macro as well.
module CJulia
using CBinding
@cbindings "./path/to/libjulia.so" begin
@ctypedef jl_nullable_float64_t @cstruct {
hasvalue::UInt8
value::Cdouble
}
@ctypedef jl_value_t @cstruct _jl_value_t
@cextern jl_gc_enable(on::Cint)::Cint
@cextern jl_gc_is_enabled()::Cint
@cextern jl_gc_alloc_0w()::Ptr{jl_value_t}
@cextern jl_gc_alloc_1w()::Ptr{jl_value_t}
@cextern jl_gc_alloc_2w()::Ptr{jl_value_t}
@cextern jl_gc_alloc_3w()::Ptr{jl_value_t}
@cextern jl_gc_allocobj(sz::Csize_t)::Ptr{jl_value_t}
@cextern jl_base_module::Ptr{@cstruct jl_module_t}
end
end