Fixed GID assignment to transmitters involved in multiple connectivity sets #15
Conversation
There was a problem hiding this comment.
This algorithm creates a (N ^ 2) / 2 "triangle" of work: iteration 2 checks everything of iteration 1, iteration 3 checks all the works of iteration 1 and 2, iteration 4 checks everything of iteration 1, 2 and 3, and so on...
If you sort and concatenate all the data based on conn_model.pre_type at the start this extra work is avoided and you do not need to do any checking of the previously existing transmitters or updating/inserting any new data.
This approach may be advantageous for the peak memory consumption.
I propose to remove all the new code and instead do this:
pre_types = set(cs.pre_type for cm in simulation.get_connectivity_sets().values())
for pre_type in pre_types:
data = []
for cm, cs in simulation.get_connectivity_sets().items():
if cs.pre_type != pre_type:
continue
pre, _ = cs.load_connections().as_globals().all()
data.append(pre[:, :2])
all_cm_transmitters = np.unique(data, axis=0)
# ... continue the algorithm like before, now that we have joined ALL the relevant data togetherPS: I've noticed np.concatenate is kind of slow, so this utility function can help:
def better_concat(items):
if not items:
raise RuntimeError('Can not concat 0 items')
l = sum(len(x) for x in items)
r = np.empty((l, items[0].shape[1]), dtype=items[0].dtype)
ptr = 0
for x in items:
r[ptr:ptr+len(x)] = x
ptr += len(x)
return r
|
The tests also show that somehow you're now placing transmitters with duplicate GIDs. I think more tests are the only way out of this precarious mess. If you have any deadlines that depend on this code, perhaps you can use a fork with your previous code (which seemed to work? At least it didn't break the existing tests) and then at a later point you can finish this PR with the optimal implementation. |
|
@filimarc @danilobenozzo could you add at least add a couple of test cases that test correct GID assigment and transmitter placement for single/multiple conn models with overlapping cell types etc? |
…within cell type and across cs, cs with same transmitter
|
yeah.. new config files are not being found. |
|
Hi, i will publish the bsb-test package with your new config so github can dowload it with pip. Btw i tried to run test on my PC but it fails the 4chunck test. with ok
|
|
@filimarc ok now it should work |
For each connectivity set, kept track of GIDs grouped by pre-synaptic cell type.
If a pre-synaptic cell already has a GID for [cell, branch], it is re-used in the case of multiple synapses.