dat-discussions@yahoogroups.com wrote:
> Caitlin,
> I had not looked at this partial multicast case.
>
> For Send case, what is the semantic when a buffer is not
> posted by one of the receivers?
> A group "failes" and need to be recreated?
> A receiver falls out of the group?
>

My hunch is that the receiver without a buffer would
have to NACK the packet (if it anticipated that it
would soon have a buffer) and drop-out of the group
if was a persistent shortage.

Going beyond multicast to also embrace partial reliability
would further require unambiguous grouping of packets into
"exchanges". With full reliability, a gap can simply block
*all* later completions. With partial reliability you have
to correctly assign the "data missing" warning to the correct
message/completion. Theoretically doable, but not trivial.

> The meaning to R-key for sender and receivers become strange.
> Sender can create a key usable by all receivers. Model can be
> extended to be restricted to a group.
> But for receivers, unless there is a way for all of them to
> generate the same key or sender lower layer impl somehow
> create a table under the covers to convert a single key to
> receiver specific key and present a single key to a sender...
>
Yes, that is *exactly* what it implies -- each receiver
must be able to create an R-Key with the same number that
maps to the local buffer that is logically the same from
the viewpoint of the transmitter.

This is easy to specify as a protocol. But it does not
work with the very common implementation strategy where
the R-Key is a direct index to an RDMA Device accessible
resource. Indeed, *requiring* the RDMA Device to accept
a user-supplied value for the R-Key would effectively
require a layer of indirection that would cost on-chip
resources and/or extra memory references.


In any event, any attempt to develop "Multicast RDMA"
would have to show patterns across multiple protocols.
What portion of the usage of multicast is focused on
multicast synchronization vs. multicast data distribution.
What requirement best characterizes application expectations:
reliable, partially reliable or unreliable? Are groups
static, or can nodes drop out? What about dropping in?