[zeromq-dev] Problem: ipc:// does not work on Windows

Sat Oct 4 19:21:49 CEST 2014

I see two problems:

* Eventually with enough time all 1024 port will be used, what then ?
(currently we are not cleaning entries)
* Client cannot write entry because the port the client is free can be
actually taken by another not ZeroMQ application. To write an entry the
client need to check that the port is actually free.
* If the file is not sorted, is it append only? so the file will get very
big eventually.

If only the server is binding the server can know if specific entry already
retired by trying to bind to the port and if succeed the server can
override the existing entry with it address.

The client need to resolve the address on every connect try (like DNS).

Append only file will eventually get to big.

On Sat, Oct 4, 2014 at 7:34 PM, Pieter Hintjens <ph at imatix.com> wrote:

> Even on connect, the client can open the file and create the entry.
> Then the binding side will use that same entry. The connect can go off
> and try to connect to the specified port.
>
> File doesn't need any order, it's fast to scan it IMO. I'd suggest
> allowing e.g. 1024 ports. Doesn't matter, can be easily changed...
> most apps don't use huge numbers of IPC endpoints (actually, zero on
> all Windows boxes now :-)
>
> -Pieter
>
> On Sat, Oct 4, 2014 at 8:42 AM, Doron Somech <somdoron at gmail.com> wrote:
> > Only adding that I think client cannot write to the file because connect
> > cannot make sure the port is actually open, only the bind can.
> >
> > Are you suggesting file order by the port with a fix entry size or
> variable?
> > what is the size of the range?
> >
> > On Fri, Oct 3, 2014 at 3:03 PM, Pieter Hintjens <ph at imatix.com> wrote:
> >>
> >> I think both connect and bind have to check the file. It doesn't need
> >> cleaning up IMO: as long as the IPC endpoints aren't duplicated. It
> >> becomes a persistent map from endpoint to socket. Indeed, either
> >> connect or bind can create a new entry (same algorithm: lock file,
> >> look for entry, create one if it's missing).
> >>
> >> Come to think of it we don't need the hashing, if we use a locked
> >> file. Random port or sequential port, it'll all work. We should
> >> probably use a range outside (below) the normal ephemeral range (49152
> >> to 65535) to avoid conflicts with normal use. I'd suggest a block
> >> before 49152.
> >>
> >> On Fri, Oct 3, 2014 at 12:30 PM, Doron Somech <somdoron at gmail.com>
> wrote:
> >> > Ok, and the client should always check the file anyway?
> >> >
> >> > So I think we can get rid of the cleanup, but the file can be big if
> it
> >> > append only.
> >> >
> >> > I suggest we allocate fix size for each entry in the file, the first
> >> > entry
> >> > in the file will be the first port in the ephemeral port range, we
> >> > actually
> >> > don't need to write the port, only the address because the port is
> >> > indexed.
> >> >
> >> > The binding socket enter the mutex and try to bind according to the
> >> > hashed
> >> > port and increase on failure, when succeeding the server write the
> >> > address
> >> > to the correct location at the file terminated by '\0' and exit the
> >> > mutex.
> >> >
> >> > The connecting socket enter the mutex and search for the address,
> >> > starting
> >> > at the hashed location, then exit the mutex.
> >> >
> >> > The last question is do we want to hash or just use ephemeral port,
> the
> >> > advantage of hashing is quicker search for the connecting socket but
> >> > slower
> >> > binding for the server. Using ephemeral port will be a little easier
> to
> >> > implement and  quicker for the binding socket, however the connecting
> >> > socket
> >> > will have to search the entire file.
> >> >
> >> > On Thu, Oct 2, 2014 at 11:27 PM, Pieter Hintjens <ph at imatix.com>
> wrote:
> >> >>
> >> >> How about a mix of hashing and text file? That means a given endpoint
> >> >> will always use the same port by default, and then we can increment
> to
> >> >> find a free port. This removes most/all the problems of cleaning up,
> I
> >> >> think.
> >> >>
> >> >> On Thu, Oct 2, 2014 at 5:32 PM, Doron Somech <somdoron at gmail.com>
> >> >> wrote:
> >> >> > I would go with mutex and shared file, but I think it is
> complicated
> >> >> > to
> >> >> > implement, because every time socket connects it has to check the
> >> >> > file
> >> >> > and
> >> >> > cannot actually use hashing. The problems with hashing:
> >> >> >
> >> >> > * Try to bind and port is taken ?
> >> >> > * Try to connect but the peer is not zeromq
> >> >> > * Try to connect and the peer is zeromq but not the one I actually
> >> >> > want.
> >> >> >
> >> >> > Because of the third problem we will have to check the file on
> every
> >> >> > connect
> >> >> > which mean we don't need the hashing, just the file.
> >> >> >
> >> >> > So we have a file, every time we bind we bind to ephemeral port and
> >> >> > write a
> >> >> > row to the file with the port and the name and every time we
> connect
> >> >> > we
> >> >> > check the file, but the remain case is when do we clear the file?
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> > On Thu, Oct 2, 2014 at 3:12 PM, Pieter Hintjens <ph at imatix.com>
> >> >> > wrote:
> >> >> >>
> >> >> >> This is annoying mainly because it hurts code portability and also
> >> >> >> because ipc:// (especially with abstract endpoints) is a nicer
> >> >> >> semantic for local services than TCP ports. e.g. I've a protocol
> >> >> >> called zccp and the default location for a zccp broker is
> >> >> >> ipc://@/zccp.
> >> >> >>
> >> >> >> Solution: cheat, on Windows
> >> >> >>
> >> >> >> Real solution: accept "ipc://" endpoints and hash them to a socket
> >> >> >> number in the ephemeral range, then bind/connect on 127.0.0.1 with
> >> >> >> that port number.
> >> >> >>
> >> >> >> Difficulty: hash conflicts. :-)
> >> >> >>
> >> >> >> If anyone has a plausible solution to hash conflicts (registry?)
> >> >> >> then
> >> >> >> this should be quite simple to make.
> >> >> >>
> >> >> >> -Pieter
> >> >> >> _______________________________________________
> >> >> >> zeromq-dev mailing list
> >> >> >> zeromq-dev at lists.zeromq.org
> >> >> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >> >> >
> >> >> >
> >> >> >
> >> >> > _______________________________________________
> >> >> > zeromq-dev mailing list
> >> >> > zeromq-dev at lists.zeromq.org
> >> >> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >> >> >
> >> >> _______________________________________________
> >> >> zeromq-dev mailing list
> >> >> zeromq-dev at lists.zeromq.org
> >> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >> >
> >> >
> >> >
> >> > _______________________________________________
> >> > zeromq-dev mailing list
> >> > zeromq-dev at lists.zeromq.org
> >> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >> >
> >> _______________________________________________
> >> zeromq-dev mailing list
> >> zeromq-dev at lists.zeromq.org
> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> >
> >
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20141004/885d7174/attachment.htm>