python - How to get chunks of elements from a queue? -

i have queue need chunks of 10 entries , put them in list, processed further. code below works (the "processed further" is, in example, print list out).

import multiprocessing  # example of actual queue q = multiprocessing.queue() in range(22):     q.put(i) q.put("end")  counter = 0 mylist = list() while true:     v = q.get()     if v == "end":         # outputs incomplete (< 10 elements) list         print(mylist)         break     else:         mylist.append(v)         counter += 1         if counter % 10 == 0:             print(mylist)             # empty list             mylist = list()  # outputs # [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] # [10, 11, 12, 13, 14, 15, 16, 17, 18, 19] # [20, 21] 

this code ugly. not see how improve - read time ago how use iter sentinel fail see how problem make use of it.

is there better (= more elegant/pythonic) way solve problem?

you use iter twice: iter(q.get, 'end') returns iterator can iterate on values in queue until 'end' returned q.get().

then use the grouper recipe

iter(lambda: list(it.islice(iterator, 10)), []) 

to group iterator chunks of 10 items.

import itertools import multiprocessing mp  q = mp.queue() in range(22):     q.put(i) q.put("end")  iterator = iter(q.get, 'end') chunk in iter(lambda: list(it.islice(iterator, 10)), []):     print(chunk) 

yields

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9] [10, 11, 12, 13, 14, 15, 16, 17, 18, 19] [20, 21]