Friday, March 18, 2011

SQS Queues and more Simple DB

Apparently I need to send json to an SQS queue. It's described at http://globalconstant.scnay.com/2011/03/02/programming-sqs-with-python-and-boto/, and http://classes.windley.com/462/wiki/index.php/SQS_Queues.
So I replaced the necessary fields with content specific to the 'commentprocess' in sqs. Still, while it doesn't throw any errors, I'm not seeing my messages being posted into the queue, so I'm not sure if it's working. Yuck
----------------------------------------------------
SO I came back to it, didn't change anything, just ran it again and saw my comment was added to the queue. I think it was just getting popped out of the queue before it was being read before.
So now, I have to add the comment to simple db. I do that in just the same way as I saved new ratings, as shown on http://cloudcarpenters.com/blog/simpledb_primer_with_python_and_boto/.
-----------------------------------------------------
The next part of teh lab required me to again add something to the simple db database, but teh catch was that it's an image and I needed to inform the db of the image's height and width. Well, all I was given was a url! How should I know?!
But a quick google search revealed a forum with the answer (the first reply didn't actually work, as noted later in the thread, but teh second did): http://osdir.com/ml/python.image/2004-04/msg00052.html
-------------------------------------------------------
Ok so now I'm adding a daemon to run every few seconds to process messages in the comment or image queues, and according to what those messages say, I'll approve comments and images.
I'm following the instructions here:http://www.jejik.com/articles/2007/02/a_simple_unix_linux_daemon_in_python/
I downloaded the two files, put themin the same folder, and they work basically as is. (One difficult thing though: the daemon (I suppoes obviously) runs in a different process than the shell from which it was called, so you can't print out debug lines to it. I suppose I could configure it to still output to the same logfile I've been using, but that requires extra learning. No, for now I'm just going to get the code to work in a function where I can see the output and logs, and then paste it into the daemon function's "run" method.
Now I need to read and pop from the queues.
I just noticed on the lab's page, at the very bottom, that there's an endpoint (a url I call and get a result from) to

Wednesday, March 2, 2011

simple db and django

I'm working on Project3 from my cs 462 class, where basically we need to make an 'appserver' which returns json stuff.
Rather than making my machine listen on port 8010 for the appserver and seperately on 80 for the regular webserver, I just got apache to listen on port 8010, and reused all my code from the webserver. (I know, pretty lazy eh. But I'm in a bit of a rush). Followed instructions on... actually I can't find it. Google "apache listen different port", but this site has ok instructions: http://httpd.apache.org/docs/2.0/vhosts/examples.html. (Basically: find the ports.conf file, and change "listen 80" to "listen 8010").
----------------------------------------------
So now I'm trying to add stuff to simple db. I receive a post from another page, and then add teh content of the post ot the db. I'm following the instructions on http://cloudcarpenters.com/blog/simpledb_primer_with_python_and_boto/.
Done: I know it's working a bit.
I was getting big images instead of nice thumbnails. I found I was using the wrong url. needed to get images from amazon.com/...[imagekey]t.jpg. I was using the original image url.
ratesubmit. In the db it's stored as an int with a max of 500, but we want to store it as a float with a max of 5.0. So, I created a function which simply divides all the ratings by 100 and turns them into floats with the following assignment:
image['rating'] = float(image['rating'])/100
Still, rating not actualy getting inputted. And it's difficult to debug the page I visit sends a post to a page, which sends a post on teh server-side to teh page I want ot debug. I'm trying to user poster to send a post to that url, but it's not working. Maybe I'll just make my own form... Ok, so I used firebug (a better firefox plugin), made my own form in it with





Found some wacky stuff. I'm notious from programming so much. weee!
And was able to see the error on that page directly.
-view image isn't working quite right. app expects a get parameter, but the request is coming in the form of http://blah.thing/[variablehere], not http://blah.thing/?variable=[variablehere]. Luckily, I know django can handle this easily, I just don't remember what this functionality is called
Logging for this project was super easy. You just send a post to a url with the info it wanted (I used python's urlib, and urlib2, with teh commands urllib2.Request('http://imaj.lddi.org:8080/log/submit',urllib.urlencode(params)), and urllib2.urlopen(newReq))

-save my personal testing webserver (make appserver's url a variable)

I realized I forgot to implement part of gettin ga list of images. Specifically, I always get the set of first images.
SOoo... boy, I find this very difficult to just think right now. yikes.
cannot compute. DOES NOT COMPUUUTE!
if ratesort<=nextratesort

--------------------------------------
ok now it's all working, except the registering for the load balancer. Somehow my file which call the bash scrip tot restart it isn't working. I htink I'll just directly get the python userscript ot restart the server