neater build system

This commit is contained in:
Aaron Manning 2022-01-20 20:07:42 +11:00 committed by aaron-jack-manning
parent d295103284
commit a8cdd2411f
16 changed files with 157 additions and 83 deletions

View File

@ -4,20 +4,25 @@ This repository contains my custom OCaml standard library and build system.
## Disclaimer: ## Disclaimer:
This is very bespoke for my requirements, and only something I use when writing code to be read and used by myself. In general, I do not recommend doing something like this. This repository exists because it shows off how one can effectively compile an OCaml project without the standard library but still expose the few functions they may need, a task made remarkably difficult, and because my implementations of some data structures and algorithms may be useful to people new to functional programming. This is very bespoke for my requirements, and only something I use when writing code to be read and used by myself. In general, I do not recommend doing something like this. This repository exists because it shows off how one can effectively compile an OCaml project without the standard library but still expose the few functions they may need, a task made remarkably difficult by the way the compiler works, and because my implementations of some data structures and algorithms may be useful to people new to functional programming.
Also note that I use WSL for all OCaml programming, and this is my recommendation for Windows users. The only software dependencies for running and debugging code in this project are the OCaml compiler and make.
## Modules ## Modules
This library includes the following custom modules: This library includes the following custom modules:
- Int (exposure of basic integer arithmetic functions) - Bool (for basic operations on booleans)
- Float (exposure of basic float arithmetic functions) - Char (for basic operations on characters)
- Option (functions for working with the option monad) - Float (for basic operations on floating point numbers)
- Stack (functional stack data structure) - Int (for basic operations on integers)
- List (functional list data structure) - List (functional list data structure)
- Map (functional map implemented as a red-black tree) - Map (functional map implemented as a red-black tree)
- Option (functions for working with the option monad)
- Queue (functional queue implemented as two lists) - Queue (functional queue implemented as two lists)
- Set (functional set implemented as a red-black tree) - Set (functional set implemented as a red-black tree)
- Stack (functional stack data structure)
- String (for basic operations on strings)
- Tree (functional generic tree type with some general functions to manipulate it) - Tree (functional generic tree type with some general functions to manipulate it)
## Exposure of Functions from Standard Library ## Exposure of Functions from Standard Library
@ -32,52 +37,48 @@ Some functions exist in the `FromStdlib` module which have names starting with `
In order to prevent duplicate definitions of common types like collections, but still allow things like list literals to work, and to prevent the need of a type annotation at the module level, a `Exposed` module is provided to be opened in code files which exposes types like `'a queue` and other collections. `Exposed` also includes generic operators that should be opened file wide, such as function composition (`>>`). This should always be opened at the top of project files. In order to prevent duplicate definitions of common types like collections, but still allow things like list literals to work, and to prevent the need of a type annotation at the module level, a `Exposed` module is provided to be opened in code files which exposes types like `'a queue` and other collections. `Exposed` also includes generic operators that should be opened file wide, such as function composition (`>>`). This should always be opened at the top of project files.
## Build Process ## Compiler Options
Since I wanted to compile with some aspects of the Standard library (or import the source files separately), the build process is a little complicated. Two makefiles are included, one within the `lib` folder (which should not be edited) and one at the top level which calls commands in the makefile within `lib`. From the top level, the following commands should be used to build and manage projects: I typically compile everything with the `-O3` flag for flambda optimization, and this can obviously be changed depending on requirements, as can the use of `ocamlopt` instead of `ocamlc` but if that is changed the final linking will need to be done with `.cmo` instead of `.cmx` files.
- `make build` to build all files, including main. ## Building the Standard Library
- `make clean` which removes all auto-generated files, leaving only source code behind.
- `make mostlyclean` which removes all auto-generated files, except `program`, the final executable.
- `make run` which runs the executable `program`, created by `make build`.
In general, a successful build process would be: Within the `lib` folder a makefile is included so that the standard library can be built using `make build`. This will generate a file called `library.cmxa` which can then be used in projects as described in the next section. In general, build the library once and do not clean it before creating a larger project.
## Creating a Project Using This Library
In the root folder of this repository, a makefile is provided to be used for projects that make use of this standard library.
To create a new project, simply create the file you wish to add in the top level. For example, let's say we are creating a file called `main.ml`.
We first create the file in the root directory, and then change the build section of makefile from this:
``` ```
$ make build build:
$ make mostlyclean ocamlopt $(CUSTOM_LIBRARY_LOCATION)/library.cmxa -o program
$ make run
``` ```
Other commands exist to clean just the project files or just the standard library. These are `make topmostlyclean`, `make topclean` and `make stdlibclean`. to this:
Also take note of the fact that I typically compile everything with `-S` and `-O3` for assembly code files and flambda optimization correspondingly, and this can obviously be changed depending on requirements, as can the use of `ocamlopt` instead of `ocamlc` but if that is changed the final linking will need to be done with `.cmo` instead of `.cmx` files.
## Adding New Modules to the Project
To add new modules to the project, I recommend following the method used with `main.ml` in the top level makefile.
In that case, there are three parts to successful compilation. The first two are specific to the module:
``` ```
$(GENERATE_MLI) main.ml > main.mli build:
$(COMPILE) main.ml
ocamlopt $(CUSTOM_LIBRARY_LOCATION)/library.cmxa main.cmx -o program
``` ```
which autogenerates the `.mli` file. Obviously if the `.mli` should not be autogenerated and instead authored individually, as is recommended in most cases, omit this line. Any additional files that need to be added can be added in the same way, by adding the line to compile and adding the `.cmx` file to the final step.
If adding in other files, order them within both places according to the desired file order, and if adding an `.mli` file with the `.ml` file, simply compile with the `.mli` file as well, before the `.ml` file. For example:
``` ```
$(COMPILE) main.mli main.ml $(COMPILE) main.mli main.ml
``` ```
while compiles the `.mli` and `.ml` files with the standard flags. This leaves a project which can be built with `make build`, run with `make run` and cleaned with `make clean`. `make install` is also an implemented command which builds the project and cleans all except the executable.
Then, any new files which are added to the compilation steps need to be included in the linking, by adding the lowercased module name, followed by a `.cmx` extension, before `main.cmx` in the following line:
``` ## Adding New Modules to the Standard Library
ocamlopt -O3 $(STDLIB_FILES) main.cmx -o program
```
## Adding New Modules to the Library
All new files added to the library need an `.ml` and `.mli` file in the `lib` folder. All new files added to the library need an `.ml` and `.mli` file in the `lib` folder.
@ -87,21 +88,26 @@ Once the files exist, the compilation step needs to be added to the `makefile` w
$(STANDARD_COMPILE) newModule.mli newModule.ml $(STANDARD_COMPILE) newModule.mli newModule.ml
``` ```
Then the file needs to be added to the list of standard library files for the top level `makefile` to find them. This is the first variable named `STDLIB_FILES`. All references are preceeded by `lib/` so they can be found within the correct folder and use the `.cmx` extension, as only the compiled files need to be linked. Then the final step within `build` also needs to be altered to include the corresponding `.cmx` file.
```
ocamlopt -a fromStdlib.cmx exposed.cmx int.cmx float.cmx option.cmx stack.cmx list.cmx map.cmx queue.cmx set.cmx tree.cmx string.cmx newModule.cmx -o $(LIB_NAME).cmxa
```
Just as with the project in the top level, file order should be consistent across compile lines and this final line.
## The Core Library ## The Core Library
One of the unfortunate consequences of the way OCaml's compilation works, is that there is a library called the core library (not to be confused with Jane Street's Core), documented [here](https://ocaml.org/manual/core.html), which contains some definitions for types and exceptions, yet does not include the code from the stdlib that uses them. When compiling with the `-nopervasives` flag, this is still included but without the standard library. While this makes sense from the perspective of having some fundamental exceptions always available, having types like `list` included makes it very annoying when implemented a custom standard library. This quirk is why my library has no type definition for `list`, `bool`, `option`, etc. but still uses these types. One of the unfortunate consequences of the way OCaml's compilation works, is that there is a library called the core library (not to be confused with Jane Street's Core), documented [here](https://ocaml.org/manual/core.html), which contains some definitions for types and exceptions, yet does not include the code from the stdlib that uses them. When compiling with the `-nopervasives` flag, this is still included but without the standard library. While this makes sense from the perspective of having some fundamental exceptions always available, having types like `list` included makes it very annoying when implemented a custom standard library. This quirk is why my library has no type definition for `list`, `bool`, `option`, etc. but still uses these types.
## Remaining to Expose from Actual Standard Library ## Planned Changes
The following modules include functions that I still intend on adding by exposing them from the actual OCaml standard library, but haven't gotten around to doing it yet: The following modules are some that I plan on introducing in future iterations of this project:
- Array - Array: for operations with mutable arrays.
- Scanf - Random: for random number generation.
- Random - File: for file IO.
- Input: for command line inputs and arguments.
In addition to these, I plan on constructing a file IO library, from the functions in the existing standard library such as `open_in` and `close_in` for example (which are in `pervasives.ml` in the original library).
## Tests ## Tests

3
lib/bool.ml Normal file
View File

@ -0,0 +1,3 @@
open FromStdlib open Exposed
let of_string = stdlib_bool_of_string_opt

2
lib/bool.mli Normal file
View File

@ -0,0 +1,2 @@
(** Converts the string to a bool, returning option type to account for invalid strings. *)
val of_string : string -> bool option

3
lib/char.ml Normal file
View File

@ -0,0 +1,3 @@
open FromStdlib open Exposed
let of_int = stdlib_char_of_int

2
lib/char.mli Normal file
View File

@ -0,0 +1,2 @@
(** Converts an int to a char. *)
val of_int : int -> char

View File

@ -1,6 +1,11 @@
open FromStdlib open Exposed open FromStdlib open Exposed
let ( + ) a b = stdlib_plus_float a b let ( + ) a b = stdlib_plus_float a b
let ( - ) a b = stdlib_minus_float a b let ( - ) a b = stdlib_minus_float a b
let ( * ) a b = stdlib_multiply_float a b let ( * ) a b = stdlib_multiply_float a b
let ( / ) a b = stdlib_divide_float a b let ( / ) a b = stdlib_divide_float a b
let of_string = stdlib_float_of_string_opt

View File

@ -5,3 +5,6 @@ val ( - ) : float -> float -> float
val ( * ) : float -> float -> float val ( * ) : float -> float -> float
val ( / ) : float -> float -> float val ( / ) : float -> float -> float
(** Converts the string to a float, returning option type to account for invalid strings. *)
val of_string : string -> float option

View File

@ -4,6 +4,20 @@ let failwith = Stdlib.failwith
let printf = Printf.printf let printf = Printf.printf
external ( |> ) : 'a -> ('a -> 'b) -> 'b = "%revapply" external ( |> ) : 'a -> ('a -> 'b) -> 'b = "%revapply"
external ignore : 'a -> unit = "%ignore"
external stdlib_int_of_char : char -> int = "%identity"
let stdlib_char_of_int = char_of_int
let stdlib_string_of_bool = string_of_bool
let stdlib_bool_of_string = bool_of_string
let stdlib_bool_of_string_opt = bool_of_string_opt
let stdlib_string_of_int = string_of_int
external stdlib_int_of_string : string -> int = "caml_int_of_string"
let stdlib_int_of_string_opt = int_of_string_opt
let stdlib_string_of_float = string_of_float
external stdlib_float_of_string : string -> float = "caml_float_of_string"
let stdlib_float_of_string_opt = float_of_string_opt
external stdlib_plus_int : int -> int -> int = "%addint" external stdlib_plus_int : int -> int -> int = "%addint"
external stdlib_minus_int : int -> int -> int = "%subint" external stdlib_minus_int : int -> int -> int = "%subint"
@ -43,3 +57,6 @@ external ( >= ) : 'a -> 'a -> bool = "%greaterequal"
external not : bool -> bool = "%boolnot" external not : bool -> bool = "%boolnot"
external ( or ) : bool -> bool -> bool = "%sequor" external ( or ) : bool -> bool -> bool = "%sequor"
external ( & ) : bool -> bool -> bool = "%sequand" external ( & ) : bool -> bool -> bool = "%sequand"

View File

@ -1,23 +1,43 @@
val failwith : string -> 'a val failwith : string -> 'a
val printf : ('a, out_channel, unit) format -> 'a val printf : ('a, out_channel, unit) format -> 'a
external ( |> ) : 'a -> ('a -> 'b) -> 'b = "%revapply" external ( |> ) : 'a -> ('a -> 'b) -> 'b = "%revapply"
external ignore : 'a -> unit = "%ignore"
external stdlib_int_of_char : char -> int = "%identity"
val stdlib_char_of_int : int -> char
val stdlib_string_of_bool : bool -> string
val stdlib_bool_of_string : string -> bool
val stdlib_bool_of_string_opt : string -> bool option
val stdlib_string_of_int : int -> string
external stdlib_int_of_string : string -> int = "caml_int_of_string"
val stdlib_int_of_string_opt : string -> int option
val stdlib_string_of_float : float -> string
external stdlib_float_of_string : string -> float = "caml_float_of_string"
val stdlib_float_of_string_opt : string -> float option
external stdlib_plus_int : int -> int -> int = "%addint" external stdlib_plus_int : int -> int -> int = "%addint"
external stdlib_minus_int : int -> int -> int = "%subint" external stdlib_minus_int : int -> int -> int = "%subint"
external stdlib_multiply_int : int -> int -> int = "%mulint" external stdlib_multiply_int : int -> int -> int = "%mulint"
external stdlib_divide_int : int -> int -> int = "%divint" external stdlib_divide_int : int -> int -> int = "%divint"
external stdlib_mod_int : int -> int -> int = "%modint" external stdlib_mod_int : int -> int -> int = "%modint"
external stdlib_plus_float : float -> float -> float = "%addfloat" external stdlib_plus_float : float -> float -> float = "%addfloat"
external stdlib_minus_float : float -> float -> float = "%subfloat" external stdlib_minus_float : float -> float -> float = "%subfloat"
external stdlib_multiply_float : float -> float -> float = "%mulfloat" external stdlib_multiply_float : float -> float -> float = "%mulfloat"
external stdlib_divide_float : float -> float -> float = "%divfloat" external stdlib_divide_float : float -> float -> float = "%divfloat"
external stdlib_string_length : string -> int = "%string_length" external stdlib_string_length : string -> int = "%string_length"
val stdlib_string_concat : string -> string -> string val stdlib_string_concat : string -> string -> string
external ( = ) : 'a -> 'a -> bool = "%equal" external ( = ) : 'a -> 'a -> bool = "%equal"
external ( <> ) : 'a -> 'a -> bool = "%notequal" external ( <> ) : 'a -> 'a -> bool = "%notequal"
external ( < ) : 'a -> 'a -> bool = "%lessthan" external ( < ) : 'a -> 'a -> bool = "%lessthan"
external ( > ) : 'a -> 'a -> bool = "%greaterthan" external ( > ) : 'a -> 'a -> bool = "%greaterthan"
external ( <= ) : 'a -> 'a -> bool = "%lessequal" external ( <= ) : 'a -> 'a -> bool = "%lessequal"
external ( >= ) : 'a -> 'a -> bool = "%greaterequal" external ( >= ) : 'a -> 'a -> bool = "%greaterequal"
external not : bool -> bool = "%boolnot" external not : bool -> bool = "%boolnot"
external ( or ) : bool -> bool -> bool = "%sequor" external ( or ) : bool -> bool -> bool = "%sequor"
external ( & ) : bool -> bool -> bool = "%sequand" external ( & ) : bool -> bool -> bool = "%sequand"

View File

@ -1,8 +1,16 @@
open FromStdlib open FromStdlib
let ( + ) a b = stdlib_plus_int a b let ( + ) a b = stdlib_plus_int a b
let ( - ) a b = stdlib_minus_int a b let ( - ) a b = stdlib_minus_int a b
let ( * ) a b = stdlib_multiply_int a b let ( * ) a b = stdlib_multiply_int a b
let ( / ) a b = stdlib_divide_int a b let ( / ) a b = stdlib_divide_int a b
let ( mod ) a b = stdlib_mod_int a b let ( mod ) a b = stdlib_mod_int a b
let of_char = stdlib_int_of_char
let of_string = stdlib_int_of_string_opt

View File

@ -7,3 +7,9 @@ val ( * ) : int -> int -> int
val ( / ) : int -> int -> int val ( / ) : int -> int -> int
val ( mod ) : int -> int -> int val ( mod ) : int -> int -> int
(** Converts the char to an int. *)
val of_char : char -> int
(** Converts the string to an int, returning option type to account for invalid strings. *)
val of_string : string -> int option

View File

@ -1,8 +1,8 @@
STANDARD_FLAGS = -S -O3 STANDARD_FLAGS = -O3
STANDARD_COMPILE = ocamlopt $(STANDARD_FLAGS) -nopervasives -c STANDARD_COMPILE = ocamlopt $(STANDARD_FLAGS) -nopervasives -c
LIB_NAME = library
# compiles the entire custom standard library build:
compile:
# fromStdlib manages things that need to be exposed from the standard library # fromStdlib manages things that need to be exposed from the standard library
ocamlopt $(STANDARD_FLAGS) -c fromStdlib.mli fromStdlib.ml ocamlopt $(STANDARD_FLAGS) -c fromStdlib.mli fromStdlib.ml
@ -20,7 +20,11 @@ compile:
$(STANDARD_COMPILE) set.mli set.ml $(STANDARD_COMPILE) set.mli set.ml
$(STANDARD_COMPILE) tree.mli tree.ml $(STANDARD_COMPILE) tree.mli tree.ml
$(STANDARD_COMPILE) string.mli string.ml $(STANDARD_COMPILE) string.mli string.ml
$(STANDARD_COMPILE) char.mli char.ml
$(STANDARD_COMPILE) bool.mli bool.ml
ocamlopt -a fromStdlib.cmx exposed.cmx int.cmx float.cmx option.cmx stack.cmx list.cmx map.cmx queue.cmx set.cmx tree.cmx string.cmx -o $(LIB_NAME).cmxa
# clean removes all except source files. autogenerated mli files are also removed.
clean: clean:
rm -f *.o *.a *.s *.cmi *.cmx *.cmxa *.cmo *.cma rm -f *.o *.a *.s *.cmi *.cmx *.cmxa *.cmo *.cma

View File

@ -3,3 +3,9 @@ open FromStdlib open Exposed
let ( + ) = stdlib_string_concat let ( + ) = stdlib_string_concat
let length = stdlib_string_length let length = stdlib_string_length
let of_int = stdlib_string_of_int
let of_float = stdlib_string_of_float
let of_bool = stdlib_string_of_bool

View File

@ -3,3 +3,12 @@ val ( + ) : string -> string -> string
(** Calculates the length of the provided string. *) (** Calculates the length of the provided string. *)
val length : string -> int val length : string -> int
(** Converts an int to a string. *)
val of_int : int -> string
(** Converts a float to a string. *)
val of_float : float -> string
(** Converts a boolean to a string. *)
val of_bool : bool -> string

View File

@ -1,3 +0,0 @@
open FromStdlib open Exposed
let _ = printf "Hello, World\n"

View File

@ -1,35 +1,18 @@
STDLIB_FILES = lib/fromStdlib.cmx lib/exposed.cmx lib/int.cmx lib/float.cmx lib/option.cmx lib/stack.cmx lib/list.cmx lib/map.cmx lib/queue.cmx lib/set.cmx lib/tree.cmx lib/string.cmx CUSTOM_LIBRARY_LOCATION = lib
COMPILE = ocamlopt -O3 -nopervasives -I $(CUSTOM_LIBRARY_LOCATION) -c
GENERATE_MLI = ocamlopt -i -I lib
COMPILE = ocamlopt -O3 -nopervasives -I lib -c
build: build:
# compile the standard library by using the internal makefile ocamlopt $(CUSTOM_LIBRARY_LOCATION)/library.cmxa -o program
cd lib; make compile
# main is the main file to run code in
$(GENERATE_MLI) main.ml > main.mli
$(COMPILE) main.mli main.ml
# after all files are individually compiled with -nopervasives, this is compiled with it so that fromStdlib has the necessary linking
ocamlopt -O3 $(STDLIB_FILES) main.cmx -o program
topmostlyclean:
rm -f *.o *.a *.s *.cmi *.cmx *.cmxa *.cmo *.cma main.mli
topclean:
rm -f *.o *.a *.s *.cmi *.cmx *.cmxa *.cmo *.cma main.mli program
stdlibclean:
cd lib; make clean
clean:
make topclean
make stdlibclean
mostlyclean: mostlyclean:
make topmostlyclean rm -f *.o *.a *.s *.cmi *.cmx *.cmxa *.cmo *.cma main.mli
make stdlibclean
clean:
rm -f *.o *.a *.s *.cmi *.cmx *.cmxa *.cmo *.cma main.mli program
make install:
make build
make mostlyclean
run: run:
./program ./program