[zeromq-dev] Idiot guide to update mlm server project
matjaz.ostroversnik at gmail.com
Sat Apr 9 21:00:07 CEST 2016
Thanks for quick answer(s). Please find my feedback within your mail.
On 9.4.2016 12:24, Pieter Hintjens wrote:
> On Fri, Apr 8, 2016 at 9:43 PM, Matjaž Ostroveršnik
> <matjaz.ostroversnik at gmail.com> wrote:
>> I managed to locally extend the mlm server with:
>> - ability to make out of source builds for cmake
>> - support for blobs for service interface
>> - curve support
> Nice stuff :)
> I'm writing a book on how the development process for pieces like
> Malamute works, at scalable-c.com. It's still in progress, but what
> there already is may help.
I'll check. You are really writing a lot. :-)
> There are a few ground rules. You've been editing files that were
> generated, and they quite clearly say so, in their header. When you do
> this, you are writing throw-away code.
> This is always the tradeoff with code generation. More power, yet
> harder to learn.
I am aware of benefits of the code generation.
I developed a generator for database abstraction layer. The code it
generates is in use with 80+ customers within financial industry. Few
implementations are important even on the state level (in different
"Source" and generated files in our solution are clearly separated and
in different folders. Generated files are all the same (e.g. generated
source files or let me say object file produced by the compiler) from a
build process point of view.
The build procedure uses this generator, but when you say make (i.e.
cmake) everything is within the build procedure (e.g. source file
generation, compilation, linking etc.). In cmake file of mlm I saw only
compiling of the sources but without source generation.
What is the top level build command in mlm case, which builds everything
from sources to binaries and tests data?
With mlm one needs to dig into generated sources and read the comments
(I admit the are clearly marked). The problem is that at least one
(maybe more) has two hops or more. (header file->api file->...) . Second
hop was one too much for me. ;-)
I break the rule by intention (i.e. by updating the generated file),
since I wanted to experiment. The experiment is over and I'd like to put
the changes in proper places. That's the point where I am now, and
unfortunately I am stuck.
> You need to learn two main tools: zproto and zproject. Each has a set
> of code generators, each producing a set of outputs from one or more
> input "model" files. It is quite flat, little abstraction, so easy to
> E.g. to change the client state machine you edit mlm_client.xml. (And
> again, it says this clearly in the header of mlm_client_engine.inc!)
> and then you run "make code" to regenerate.
> The second thing is, one change at a time. Start with the simplest
> change, get a PR, learn the process, and then move to the next change.
> Don't come to a project with a large change set, or people will
> probably refuse it (and that's in our C4 process).
That was my initial intention (i.e. I changed the CMake file; just few
> If you find a generated file that does not clearly state its origin,
> tell us and we'll fix it.
I completely understand your position, but please do not forget, that
you conceived, designed and gradually implemented the build procedure
like it is now, and everything is clear and understandable / explainable
for you. Have in mind poor rookies like me, who do not have such prior
Can you (actually any of the product veterans): in form of short build
instructions(part of the distribution)
- provide a list of "source" generated files
- with each file from this list provide additional data:
- what is the ultimate source for the file (we saw that some has more
than one hop)
- what is the command to convert the ultimate source file into this file
You can go also in inverted direction (i.e. A generates B,C,D, where
A->B uses program P1, A->C is using program P2, etc)
Then it is on rookie to find the description of the generator language
to be able to fix the A in a proper way. ;-)
If the sources and generated files are clearly separated and completely
integrated into the build procedure, then everything above in less
It would be nice if the build procedure:
- would be completely inside the project structure (I understand that
one building the mlm has to work with zproject project files also;
correct me please if I am not right)
- all build time generated files (e.g. generated sources, object files,
...) separated and not mixed with real sources like they are now.
This way newbies like me would have much lower learning curve.
BTW for second line above I am more that willing to take cmake part of
(re)work on me.
More information about the zeromq-dev