Should I use marshal if I am getting comma errors in JSON (Python)? -


i using python dump huge data structure (multiple lists , dictionaries) , sending on socket client.

i keep getting valueerror: 'expecting ',' delimiter: line 1 column 16177 (char 16176) @ various different locations every time run program (it column 25000, or column 13000, keeps changing).

should use marshal instead of json (or pickle)? reliable format large file sizes?

i'd suggest use pickle (or cpickle if you're on python 2.x) can serialize anything, including user-defined classes. and, docs say

the marshal serialization format not guaranteed portable across python versions. because primary job in life support .pyc files, python implementers reserve right change serialization format in non-backwards compatible ways should need arise. pickle serialization format guaranteed backwards compatible across python releases.

(emphasis mine).

another advantage of pickle:

the pickle module keeps track of objects has serialized, later references same object won’t serialized again. marshal doesn’t this.

this has implications both recursive objects , object sharing. recursive objects objects contain references themselves. these not handled marshal, , in fact, attempting marshal recursive objects crash python interpreter. object sharing happens when there multiple references same object in different places in object hierarchy being serialized. pickle stores such objects once, , ensures other references point master copy. shared objects remain shared, can important mutable objects.

you can use dill if pickle fails serialize data.


Comments

Post a Comment

Popular posts from this blog

searchKeyword not working in AngularJS filter -

i writing app using angularjs on front end. want search through table word/field; 1 search box everything. according article here: http://hello-angularjs.appspot.com/searchtable can use filter: searchkeyword this. this code, , not working expected. <label>search: <input ng-model="searchkeyword"></label> <table ng-repeat="post in posts | orderby: sort | filter: searchkeyword"> <tr> <td> {{index(post)}} </td> <td> {{post.title}} </td> <td> {{post.author}} </td> <td> {{post.content}} </td> <td> {{post.date | date: "d/m/yyyy"}} </td> </tr> </table> what should do? thank you try this: controller $scope.posts = [ {title:'title one', author:'author one', content: 'content one'}, {title:'title two', author:'author two', content: 'cont...
Read more

sequelize.js - Sequelize: sort by enum cases -

i have model ( article ) field (the name type ) of type enum('source', 'translated') , return 1 article ordered type field. source articles should returned before translated articles. like: article.findone(order: [ { type: [ 'source', 'translated' ] } ]); unfortunately there appears no built-in solution sorting enums. instead have sort by case , couldn't find documentation on how sequelize. what conventional approach here? sorting case hard in sequelize, sort field (the second answer in linked post) should possible: order: sequelize.fn('field', sequelize.col('type'), 'source', 'translated') it's supported mysql far know though
Read more

user interface - how to replace an ongoing process of image capture from another process call over the same ImageLabel in python's GUI TKinter -

i trying use following code create interactive gui changes color-space(hsv, rbg, grayscale etc) on event of button click. display opencv video in tkinter using multiprocessing being new python having problems multiprocessing , attempts on making gui change it's color space on button clicks hangs entire system. on it's implementation highly appreciated. thankyou. below code: import cv2 pil import image,imagetk import tkinter tk import numpy np multiprocessing import process , queue def quit_it(root,process): root.destroy() process.terminate() def black_andwhite(root,process): process.terminate p=process(target=capture_image, args=(5,queue, )) p.start() root.after(0, func=lambda: update_all(root, image_label, queue)) def update_image(image_label, queue): frame = queue.get() = image.fromarray(frame) b = imagetk.photoimage(image=a) image_label.configure(image=b) image_label._image_cache = b root.update() def update_all(root...
Read more