collaborative filtering - Spark ALS with Sparse Implicit Dataset -


i trying run movieals example spark implicit dataset , receiving error:

got 3856988 ratings 144250 users on 378937 movies. training: 3085522, test: 771466. 15/07/13 10:43:07 warn blas: failed load implementation from: com.github.fommil.netlib.nativesystemblas 15/07/13 10:43:07 warn blas: failed load implementation from: com.github.fommil.netlib.nativerefblas 15/07/13 10:43:10 warn tasksetmanager: lost task 3.0 in stage 29.0 (tid 192, 10.162.45.33): java.lang.assertionerror: assertion failed: lapack.dppsv returned 1. @ scala.predef$.assert(predef.scala:179) @ org.apache.spark.ml.recommendation.als$choleskysolver.solve(als.scala:386) @ org.apache.spark.ml.recommendation.als$$anonfun$org$apache$spark$ml$recommendation$als$$computefactors$1.apply(als.scala:1163) @ org.apache.spark.ml.recommendation.als$$anonfun$org$apache$spark$ml$recommendation$als$$computefactors$1.apply(als.scala:1124) @ org.apache.spark.rdd.pairrddfunctions$$anonfun$mapvalues$1$$anonfun$apply$41$$anonfun$apply$42.apply(pairrddfunctions.scala:700) @ org.apache.spark.rdd.pairrddfunctions$$anonfun$mapvalues$1$$anonfun$apply$41$$anonfun$apply$42.apply(pairrddfunctions.scala:700) @ scala.collection.iterator$$anon$11.next(iterator.scala:328) @ org.apache.spark.storage.memorystore.unrollsafely(memorystore.scala:277) @ org.apache.spark.cachemanager.putinblockmanager(cachemanager.scala:171) @ org.apache.spark.cachemanager.getorcompute(cachemanager.scala:78) @ org.apache.spark.rdd.rdd.iterator(rdd.scala:242) @ org.apache.spark.rdd.mappartitionsrdd.compute(mappartitionsrdd.scala:35) @ org.apache.spark.rdd.rdd.computeorreadcheckpoint(rdd.scala:277) @ org.apache.spark.rdd.rdd.iterator(rdd.scala:244) @ org.apache.spark.scheduler.resulttask.runtask(resulttask.scala:63) @ org.apache.spark.scheduler.task.run(task.scala:70) @ org.apache.spark.executor.executor$taskrunner.run(executor.scala:213) @ java.util.concurrent.threadpoolexecutor.runworker(threadpoolexecutor.java:1145) @ java.util.concurrent.threadpoolexecutor$worker.run(threadpoolexecutor.java:615) @ java.lang.thread.run(thread.java:745)  15/07/13 10:43:10 error tasksetmanager: task 12 in stage 29.0 failed 4 times; aborting job exception in thread "main" org.apache.spark.sparkexception: job aborted due stage failure: task 12 in stage 29.0 failed 4 times, recent failure: lost task 12.3 in stage 29.0 (tid 249, 10.162.45.33): java.lang.assertionerror: assertion failed: lapack.dppsv returned 1. @ scala.predef$.assert(predef.scala:179) @ org.apache.spark.ml.recommendation.als$choleskysolver.solve(als.scala:386) @ org.apache.spark.ml.recommendation.als$$anonfun$org$apache$spark$ml$recommendation$als$$computefactors$1.apply(als.scala:1163) @ org.apache.spark.ml.recommendation.als$$anonfun$org$apache$spark$ml$recommendation$als$$computefactors$1.apply(als.scala:1124) @ org.apache.spark.rdd.pairrddfunctions$$anonfun$mapvalues$1$$anonfun$apply$41$$anonfun$apply$42.apply(pairrddfunctions.scala:700) @ org.apache.spark.rdd.pairrddfunctions$$anonfun$mapvalues$1$$anonfun$apply$41$$anonfun$apply$42.apply(pairrddfunctions.scala:700) @ scala.collection.iterator$$anon$11.next(iterator.scala:328) @ org.apache.spark.storage.memorystore.unrollsafely(memorystore.scala:277) @ org.apache.spark.cachemanager.putinblockmanager(cachemanager.scala:171) @ org.apache.spark.cachemanager.getorcompute(cachemanager.scala:78) @ org.apache.spark.rdd.rdd.iterator(rdd.scala:242) @ org.apache.spark.rdd.mappartitionsrdd.compute(mappartitionsrdd.scala:35) @ org.apache.spark.rdd.rdd.computeorreadcheckpoint(rdd.scala:277) @ org.apache.spark.rdd.rdd.iterator(rdd.scala:244) @ org.apache.spark.scheduler.resulttask.runtask(resulttask.scala:63) @ org.apache.spark.scheduler.task.run(task.scala:70) @ org.apache.spark.executor.executor$taskrunner.run(executor.scala:213) @ java.util.concurrent.threadpoolexecutor.runworker(threadpoolexecutor.java:1145) @ java.util.concurrent.threadpoolexecutor$worker.run(threadpoolexecutor.java:615) @ java.lang.thread.run(thread.java:745)  driver stacktrace: @ org.apache.spark.scheduler.dagscheduler.org$apache$spark$scheduler$dagscheduler$$failjobandindependentstages(dagscheduler.scala:1266) @ org.apache.spark.scheduler.dagscheduler$$anonfun$abortstage$1.apply(dagscheduler.scala:1257) @ org.apache.spark.scheduler.dagscheduler$$anonfun$abortstage$1.apply(dagscheduler.scala:1256) @ scala.collection.mutable.resizablearray$class.foreach(resizablearray.scala:59) @ scala.collection.mutable.arraybuffer.foreach(arraybuffer.scala:47) @ org.apache.spark.scheduler.dagscheduler.abortstage(dagscheduler.scala:1256) @ org.apache.spark.scheduler.dagscheduler$$anonfun$handletasksetfailed$1.apply(dagscheduler.scala:730) @ org.apache.spark.scheduler.dagscheduler$$anonfun$handletasksetfailed$1.apply(dagscheduler.scala:730) @ scala.option.foreach(option.scala:236) @ org.apache.spark.scheduler.dagscheduler.handletasksetfailed(dagscheduler.scala:730) @ org.apache.spark.scheduler.dagschedulereventprocessloop.onreceive(dagscheduler.scala:1450) @ org.apache.spark.scheduler.dagschedulereventprocessloop.onreceive(dagscheduler.scala:1411) @ org.apache.spark.util.eventloop$$anon$1.run(eventloop.scala:48)

i unsure if due sparsity of dataset. works fine when trained explicitly. however, since dataset binary, error rate obtained explicit als model not accurate.

would possible me out? thank you, ben


Comments

Post a Comment

Popular posts from this blog

user interface - how to replace an ongoing process of image capture from another process call over the same ImageLabel in python's GUI TKinter -

i trying use following code create interactive gui changes color-space(hsv, rbg, grayscale etc) on event of button click. display opencv video in tkinter using multiprocessing being new python having problems multiprocessing , attempts on making gui change it's color space on button clicks hangs entire system. on it's implementation highly appreciated. thankyou. below code: import cv2 pil import image,imagetk import tkinter tk import numpy np multiprocessing import process , queue def quit_it(root,process): root.destroy() process.terminate() def black_andwhite(root,process): process.terminate p=process(target=capture_image, args=(5,queue, )) p.start() root.after(0, func=lambda: update_all(root, image_label, queue)) def update_image(image_label, queue): frame = queue.get() = image.fromarray(frame) b = imagetk.photoimage(image=a) image_label.configure(image=b) image_label._image_cache = b root.update() def update_all(root...
Read more

javascript - Using jquery append to add option values into a select element not working -

i'm looking dynamically populate select element names of items stored array. for reason it's not working , i'm not sure why. html: <div class="col-md-4 col-md-offset-4"> <select class="select1"> </select> , <select class="select2"> </select> </div> note i'm using bootstrap layout. here's jquery i'm using try , populate ".select1" jquery: select: function() { $.each(state.greens, function(index, value) { $(".select1").append("<option value =' " + index + " '>" + state.greens[index].name + "</option>"); }); } note function 'select' stored within object called 'display'. state.greens name of array i'm trying access. so @ end of html page call display.select(); call function. no error messages showing in console. also, saw question: appending ...
Read more

javascript - Restarting Supervisor and effect on FlaskSocketIO -

in index.html (html/javascript) have: $(document).ready(function(){ namespace = '/test'; var socket = io.connect('http://' + document.domain + ':' + location.port + namespace); socket.on('connect', function() { socket.emit('join', {room: 'venue_1'}); }); socket.on('my response', function(msg) { $('#log').append('<br>received #' + ': ' + msg.data); }); }); on server have: @socketio.on('connect', namespace='/test') def test_connect(): if session.get('venue_id'): emit('my response', {'data': 'connected'}) session.pop('venue_id', none) else: request.namespace.disconnect() @socketio.on('join', namespace='/test') def join(message): join_room(message['room']) room = message[...
Read more