[zeromq-dev] Idiot guide to update mlm server project

[Matjaž Ostroveršnik]

Pieter,

Thanks for quick answer(s). Please find my feedback within your mail.

Best regards

Matjaž

On 9.4.2016 12:24, Pieter Hintjens wrote:
> On Fri, Apr 8, 2016 at 9:43 PM, Matjaž Ostroveršnik
> <matjaz.ostroversnik at gmail.com> wrote:
>
>> I managed to locally extend the mlm server with:
>> - ability to make out of source builds for cmake
>> - support for blobs for service interface
>> - curve support
> Nice stuff :)
>
> I'm writing a book on how the development process for pieces like
> Malamute works, at scalable-c.com. It's still in progress, but what
> there already is may help.
I'll check. You are really writing a lot. :-)
> There are a few ground rules. You've been editing files that were
> generated, and they quite clearly say so, in their header. When you do
> this, you are writing throw-away code.
>
> This is always the tradeoff with code generation. More power, yet
> harder to learn.
I am aware of benefits of the code generation.
*bragging on*
I developed a generator for database abstraction layer. The code it 
generates is in use with 80+ customers within financial industry. Few 
implementations are important even on the state level (in different 
countries).
*bragging off*
"Source" and generated files in our solution are clearly separated and 
in different folders. Generated files are all the same (e.g. generated 
source files or let me say object file produced by the compiler) from a 
build process point of view.
The build procedure uses this generator, but when you say make (i.e. 
cmake) everything is within the build procedure (e.g. source file 
generation, compilation, linking etc.). In cmake file of mlm I saw only 
compiling of the sources but without source generation.

What is the top level build command in mlm case, which builds everything 
from sources to binaries and tests data?

With mlm one needs to dig into generated sources and read the comments 
(I admit the are clearly marked). The problem is that at least one 
(maybe more) has two hops or more. (header file->api file->...) . Second 
hop was one  too much for me. ;-)

I break the rule by intention (i.e. by updating the generated file), 
since I wanted to experiment. The experiment is over and I'd like to put 
the changes in proper places. That's the point where I am now, and 
unfortunately I am stuck.
> You need to learn two main tools: zproto and zproject. Each has a set
> of code generators, each producing a set of outputs from one or more
> input "model" files. It is quite flat, little abstraction, so easy to
> learn.
>
> E.g. to change the client state machine you edit mlm_client.xml. (And
> again, it says this clearly in the header of mlm_client_engine.inc!)
> and then you run "make code" to regenerate.
>
> The second thing is, one change at a time. Start with the simplest
> change, get a PR, learn the process, and then move to the next change.
> Don't come to a project with a large change set, or people will
> probably refuse it (and that's in our C4 process).
That was my initial intention (i.e. I changed the CMake file; just few 
lines). ;-)
> If you find a generated file that does not clearly state its origin,
> tell us and we'll fix it.
I completely understand your position, but please do not forget, that 
you conceived, designed and gradually implemented the build procedure 
like it is now, and everything is clear and understandable / explainable 
for you. Have in mind poor rookies like me, who do not have such prior 
knowledge.

Can you (actually any of the product veterans): in form of short build 
instructions(part of the distribution)
- provide a list of  "source" generated files
- with each file from this list provide additional data:
   - what is the ultimate source for the file (we saw that some has more 
than one hop)
   - what is the command to convert the ultimate source file into this file
You can go also in inverted direction (i.e. A generates B,C,D, where 
A->B uses program P1, A->C is using program P2, etc)
Then it is on rookie to find the description of the generator language 
to be able to fix the A in a proper way. ;-)

If the sources and generated files are clearly separated and completely 
integrated into the build procedure, then everything above in less 
important.

It would be nice if the build procedure:
- would be completely inside the project structure (I understand that 
one building the mlm has to work with zproject project files also; 
correct me please if I am not right)
- all build time generated files (e.g. generated sources, object files, 
...) separated and not mixed with real sources like they are now.

This way newbies like me would have much lower learning curve.
BTW for second line above I am more that willing to take cmake part of 
(re)work on me.