What's the best way to organize the directory structure of a big mixed domain project? (discussion)
10 21 Apr 2016 14:28 by u/starTracer
How would you set up the structure of a big (in our case we have around 2500 executables and libraries) mixed domain project that is not based on a particular third party framework.
If a standard directory structure is employed, then a powerful buildsystem would make many things easy as pie (e.g. automatic globbing of sources, creation of unit tests et.c.).
Some features that we have to support in whatever structure we end up using:
- "Modules", a logical grouping of artifacts that may share the same namespace.
- "Packages", a logical grouping of modules (possibly with its own namespace).
- Multi-language (C, C++, Java, Python, PLC)
- Multiple runtime targets (e.g. Linux, VxWorks, PPC, Intel). Also sometimes the same module share code between targets.
- Code generation (e.g. IDL, Qt-mocs, Py-bindings, custom...)
- Resources (e.g. UI glyphs, sounds)
- Documentation
- Unit tests
- Integration tests (scripts and helper-tools)
Also,
- Pros/cons of single artifact per source directory (like Maven projects)?
- With a big project like this, not everyone will want to build the whole source tree just to be able to build their module. How would you go about doing this? Build against a pre-installed header+library tree? What are the alternatives?
8 comments
1 u/JohnQCitizen 21 Apr 2016 17:19
My immediate thought is that you are either making your project way too complex, or you are in way over your head. By the time you start making projects like that, you should have a solid grasp on what works for your team.
0 u/starTracer [OP] 21 Apr 2016 19:23
Well, big projects tend to become complex just by the virtue of its size. And we are in the initial stages of the project (software wise), so I'm sounding the field for what have worked well for others.
1 u/BitterBiped 24 Apr 2016 22:44
Hi I am going to answer some of your questions, not all because I don't have all the issues you have.
This is how I structure my multi language projects. I build my projects on Windows and on Linux.
Binaries (all my compiled output ends up here. Note for cpp I also create a msc/gnu folder to store os specific binaries)
Build
Source Code (all my source code goes here)
Externals (most projects have external dependencies, be it other repository's, external library dependencies, or resources)
Generated Code
Release/Packages (any final complete packages built from the the ~/bin folder most build end up here)
Testing (finally all my unit testing ends up here)
The build steps for my project are:
Finally in regards to your question about building specific modules only.. this is what I do.
I generate a makefile for every component for each language. Then then I generate a makefile that build these sub components as groups.
Basically I have makefiles that build "projects/components" and makefiles that collectively run other makefiles that act like "solutions".
For example, if I want to compile all the java code in my project I simply go to my build folder and type:
You could achieve the same thing with ant files or maven however (which is something I do at my job... because a lot people are afraid of make files for some reason).
0 u/0x7a69 21 Apr 2016 18:03
Read up on fhs, then go in your own direction
0 u/starTracer [OP] 21 Apr 2016 19:14
What do you mean? Map the FHS structure for each SW-module or..?
0 u/0x7a69 22 Apr 2016 00:18
Well, you could if you wanted to, but that would be overkill, instead try the reverse; mapping all the modules a single FHS structure inside of a project folder.
0 u/grlldcheese 21 Apr 2016 20:12
What in the world are you building?
And please keep us updated.
This is ambitious regardless of your skill level. Good luck.
0 u/starTracer [OP] 22 Apr 2016 09:15
It is scientific instrument control software. Which means a lot of devices like motors, sensors and detectors. So the control software is full stack; from the back-end low level device control to data collection and processing to front-end user interfaces.