KICKOFF of FBAdvancedSearch FB/Kynetx App

So the next string of posts will be relating to my creation of a FB/Kynetx App tentatively called "Advanced Search." With it, users will be able to search through facebook photos, wall posts, and whatever else. It will primarily be a facebook app. I want practice using facebook-connect though, so it will also be a regular webpage which uses facebook-connect. Also, it would be cool if it integrates directly into the user's facebook page etc, so for that I will use Kynetx.
The first thing I need to do is make a plan of how to use this. And actually, the very first thing was to create a blog to track my progress, and provide a type of how-to manual for others.
THINGS TO DO WHEN MAKING MY FACEBOOK APP:

1. Get webhosting. The webhost must not add a banner on the top of the page (by editing the html on the page), as this will interfere with rendering of the Facebook app (it's acceptable for a webpage using facebook-connect, but not for a facebook app). I want the webhost to be able to run PHP (in order to use Facebook's PHP SDK), and MySQL (I'm actually not certain what I'll need a database for, but it's a good assumption that most webapps will need one). I want a webhost that's preferably free, because I won't initially be profiting off this, nor do I expect a huge number of users initially.
2. Register as a Facebook App using the Facebook Developer Application
3. Get the PHP SDK and install it. If the webhost has allowed my PHP to write to a file, installation of the SDK is easy. If not, I may need to manually enter my FB Developer id and key etc, and do some other configuration manually.
4. Test FBML works right. Try rendering one page with some fbml in it within the facebook frame
5. Create the "appserver". The appserver will be a set of ursl (endpoints) which I or others can send requests to, and which will return certain data in json format This is also the part that will usually be making requests via FQL or Facebook's API.
6. Create the FB-app pages. Create these pages using FBML, and utilizing my appserver's endpoints.
7. Also, create the Facebook-connect site which also uses the appserver
8. Lastly, create the kynetx app to add some HTML to the users' regular FB pages which interfaces with my "Advanced Search" app
9. Register as a facebook app (this will allow users to search for my app).
10. Test it with friends etc.
11. Monetize: either through advertisements on the page, charing users who use the app very often, or simply advertising myself as a facebook developer for hire, etc.
12. Read a book on how to market it and make it popular. This may require refactoring.
13. Bug-fixes/improvements. Respond to user feedback and bug-reports, and make necessary changes.

SQS Queues and more Simple DB

Apparently I need to send json to an SQS queue. It's described at http://globalconstant.scnay.com/2011/03/02/programming-sqs-with-python-and-boto/, and http://classes.windley.com/462/wiki/index.php/SQS_Queues.
So I replaced the necessary fields with content specific to the 'commentprocess' in sqs. Still, while it doesn't throw any errors, I'm not seeing my messages being posted into the queue, so I'm not sure if it's working. Yuck
----------------------------------------------------
SO I came back to it, didn't change anything, just ran it again and saw my comment was added to the queue. I think it was just getting popped out of the queue before it was being read before.
So now, I have to add the comment to simple db. I do that in just the same way as I saved new ratings, as shown on http://cloudcarpenters.com/blog/simpledb_primer_with_python_and_boto/.
-----------------------------------------------------
The next part of teh lab required me to again add something to the simple db database, but teh catch was that it's an image and I needed to inform the db of the image's height and width. Well, all I was given was a url! How should I know?!
But a quick google search revealed a forum with the answer (the first reply didn't actually work, as noted later in the thread, but teh second did): http://osdir.com/ml/python.image/2004-04/msg00052.html
-------------------------------------------------------
Ok so now I'm adding a daemon to run every few seconds to process messages in the comment or image queues, and according to what those messages say, I'll approve comments and images.
I'm following the instructions here:http://www.jejik.com/articles/2007/02/a_simple_unix_linux_daemon_in_python/
I downloaded the two files, put themin the same folder, and they work basically as is. (One difficult thing though: the daemon (I suppoes obviously) runs in a different process than the shell from which it was called, so you can't print out debug lines to it. I suppose I could configure it to still output to the same logfile I've been using, but that requires extra learning. No, for now I'm just going to get the code to work in a function where I can see the output and logs, and then paste it into the daemon function's "run" method.
Now I need to read and pop from the queues.
I just noticed on the lab's page, at the very bottom, that there's an endpoint (a url I call and get a result from) to

simple db and django

I'm working on Project3 from my cs 462 class, where basically we need to make an 'appserver' which returns json stuff.
Rather than making my machine listen on port 8010 for the appserver and seperately on 80 for the regular webserver, I just got apache to listen on port 8010, and reused all my code from the webserver. (I know, pretty lazy eh. But I'm in a bit of a rush). Followed instructions on... actually I can't find it. Google "apache listen different port", but this site has ok instructions: http://httpd.apache.org/docs/2.0/vhosts/examples.html. (Basically: find the ports.conf file, and change "listen 80" to "listen 8010").
----------------------------------------------
So now I'm trying to add stuff to simple db. I receive a post from another page, and then add teh content of the post ot the db. I'm following the instructions on http://cloudcarpenters.com/blog/simpledb_primer_with_python_and_boto/.
Done: I know it's working a bit.
I was getting big images instead of nice thumbnails. I found I was using the wrong url. needed to get images from amazon.com/...[imagekey]t.jpg. I was using the original image url.
ratesubmit. In the db it's stored as an int with a max of 500, but we want to store it as a float with a max of 5.0. So, I created a function which simply divides all the ratings by 100 and turns them into floats with the following assignment:
image['rating'] = float(image['rating'])/100
Still, rating not actualy getting inputted. And it's difficult to debug the page I visit sends a post to a page, which sends a post on teh server-side to teh page I want ot debug. I'm trying to user poster to send a post to that url, but it's not working. Maybe I'll just make my own form... Ok, so I used firebug (a better firefox plugin), made my own form in it with

Found some wacky stuff. I'm notious from programming so much. weee!
And was able to see the error on that page directly.
-view image isn't working quite right. app expects a get parameter, but the request is coming in the form of http://blah.thing/[variablehere], not http://blah.thing/?variable=[variablehere]. Luckily, I know django can handle this easily, I just don't remember what this functionality is called
Logging for this project was super easy. You just send a post to a url with the info it wanted (I used python's urlib, and urlib2, with teh commands urllib2.Request('http://imaj.lddi.org:8080/log/submit',urllib.urlencode(params)), and urllib2.urlopen(newReq))

-save my personal testing webserver (make appserver's url a variable)

I realized I forgot to implement part of gettin ga list of images. Specifically, I always get the set of first images.
SOoo... boy, I find this very difficult to just think right now. yikes.
cannot compute. DOES NOT COMPUUUTE!
if ratesort<=nextratesort

--------------------------------------
ok now it's all working, except the registering for the load balancer. Somehow my file which call the bash scrip tot restart it isn't working. I htink I'll just directly get the python userscript ot restart the server

forwarding a post in python

Wanted to run some data validation on a post request, then forward that request onto a different url.
Used urllib2, although it wasn't exactly intuitive.
In my python code, I receive an django.http Request object.
To forward the request on, I did:
|newReq=urllib2.Request('urlToFrwardTo',urllib.urlencode(req.POST))
and then
|urllib2.urlopen(newReq)

The part that weirded me out was the why I needed to urlencode the post. The post parameters don't go in the url! Why would I urlencode them?
Whatever, it didn't work unless I called 'urllib.urlencode' on the reqiest's POST parameters. Weird.

setting up django fun

Just notes on what I'm up to in trying to get django to work for my Computer Science 462 class. I find this helpful for remembering where I left off and commands etc. But you will probably find it very boring unless you try and do the same.


using tutorial at http://docs.djangoproject.com/en/dev/intro/tutorial01/
==========================
Found I needed root priviledges, so I allowed the default users, 'ubuntu' root priledges by calling
so I logged into bash as root by just typing
|sudo bash
then editing the /etc/sudoers file by typing
|visudo
and adding changing the line from
root ALL=(ALL) ALL
to
ubuntu,root ALL=(ALL) ALL
===========================
I edited the settings file settings.py, and configured stuff for sqlite3, like the tutorial suggested.
then, as it suggested, ran
python manage.py syncdb
which created the database file where I wanted it in /var/django/db/mikeServ.db
---------------------------
tried to view the schemas by typing
|.schema
with no luck. googled how to view schemas in sqlite.
Googled what the matter might have been, found I needed pysqlite2, so ran
|aptitude search pysqlite2
found it, and ran
|apt-get install python-pysqlite2
but still can't find anywhere how to use it really, nor what comes with that package
bah. I'll just assume it worked for now, even thought I can't check it. we shall see.
---------------------------
I want to use django for a class project which really doesn't require a local database, just using ajax and templates, so I skipped over the rest of the first page of teh tutorial having to do with creating objects/tables, but it seeemd good.
----------------------------
was interested in the admin pages to work, so I followed the instructions on teh next page:
added django.contrib.admin to INSTALLED_APPS in settings.py,
and ran
|python manage.py syncdb
then edited urls.py as indicated.
-----------------------------
Tried to run the /admin/ pages, but no go. I think I might need to restart the webserver:
|/etc/init.d/apache2 restart
When I visited my site (http://50.16.162.246/admin/) the next time, made some progress: got an error message! somethign about indentatino error in ursl.py, so I'll go fix taht. (Found the compy was right: forgot to remove preceding white-space when uncommenting).
Now though, I'm getting a 500 error with no real helpful input. Hmm. progress? probably. I bet the error is coming from not reaaally having the database schemas, though I ran syncdb 'n all. I should verify those exist for reals.
I foudn the problem was I don't have django stuff on my path, as explained in http://martinjansen.com/2008/10/20/django-settings-files-for-development-and-production/. I just need to add my settings.py file to the PATH, and some DJANGO_SETTINGS.
Trouble with that, some help also on http://ubuntuforums.org/showthread.php?t=125735
Found that didn't permanently solve anything. (This forum page has a good explanation of different config files like .bashrc and .bashprofile and stuff: http://ubuntuforums.org/showthread.php?t=125735).
Eventually, found an easy way to add to the PYTHONPATH is just to add a file named whatever.pth to something that already is on a the pythonpath (like /usr/local/lib/python2.6/dist-packages) and in that file write nothing but the path to whatever I want to add to the python path.
Still the DJANGO_SETTINGS_MODULE isn't found.
Then I realized I had edited the wrong .bashrc file. I edited /.bashrc, when really I should ahve edited ~/.bashrc. bah!
So edited taht file instead, restarted teh command-prompt shell, and voila!
I could verify that PYTHONPATH was set by typing in
|$PYTHONPATH
into the shell and it would print it out
However, I'm still getting a 500 error on teh admin pages.
Added a symbolic link to the admin media stuff like suggested on http://docs.djangoproject.com/en/dev/howto/deployment/modwsgi/, but that didn't resolve it.
Now I'm looking at http://code.djangoproject.com/wiki/django_apache_and_mod_wsgi
Ok, more of the same. But I'm onto something now.
I thought I really needed to see some better error logs. So I searched for where are the apache error logs, and found it's at:
/var/log/apache2/error.log, right next to the access.log (I didn't know they were seperate). I then noticed what the error seems to be:
[Sat Jan 29 16:25:55 2011] [error] [client 67.166.99.183] OperationalError: attempt to write a readonly database
meaning the sqlite file that its trying to write to is set to be read-only. Which kinda makes sense. So I'm gonna check what the files permissions are, and change them to amke them write-whatever-you-want. (Did that by going to the file, which I placed in /var/django/db/mikeServ.db and did the command chmod 777 mikeServ.db)
and BOOM BABY! Admin console!
Except I didn't create the symbolic link to the media files right.
And then I noticed it stillw asn't redirecting quite right. So I needed to edit the apache config file some more. Basically, I needed to make apache get the media files from where they're at on the server. So I added these lines to my /etc/apache2/sites-available/default file:
Alias /media/ /usr/lib/pymodules/python2.6/django/contrib/admin/media/

Order deny,allow
Options Indexes
Options FollowSymLinks
Allow from all

==================================
ok so now I'm trying to get some pages to work,
I ran
|python manage.py startapp view
|" " " list
|" " " home
to create 3 'apps' I guess that will host my files

and added those 3 things to my list of installed apps
and another issue is I'm not sure if django-admin.py is on my filepath. ...actually immediately remembered: it is, just under a different name, it's under django-admin (no '.py')
-------------------------------------------------
so I finished those views.py files, as follows:
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
# Create your views here.
from django.http import HttpResponse
from django.template import Context, loader, Template
import urllib2
import json

def popular(req):
t = loader.get_template('popular.tmpl')
return HttpResponse(t.render({}))

def recent(req):
t = loader.get_template('recent.tmpl')
response=urllib2.urlopen('http://imaj-app.lddi.org:8010/list/recent')
dictResponse=json.loads(response.read())
c = Context({'imagelist': dictResponse['images'],'nextsubmitdate':dictResponse['nextsubmitdate'],'nextratesort':dictResponse['nextratesort']})
return HttpResponse(t.render(c))
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
and then the template:
<<<<<<<<<<<<<<<<<<<<<<<,

Recent Images

{% for image in imagelist %}

{{image.description}}

Submitted by {{image.submituser}} on {{image.submitdate}}

{% endfor %}



{% if nextsubmitdate %}
More Images ...
{% endif %}
>>>>>>>>>>>>>>>>>>>>>
the ta gave the template to us originally in some 'cheetah' python template language, but converting it to django's template language was easy: the semantics (the way it works) are just the same, just silly difference in syntax.
and voila! I have a page which renders! bwahahaha!
----------------------------------
I noticed the page was missing headers, css stuff, etc.
After inspecting the templates, I realized they had no declaration of etc, or importing of css files. Reason: I wasn't using the main template at all!
Apparently Cheetah works differently from django's template. In Cheetah the parent template imports the others by just inserting $main somewhere in teh html. IN django, you need a tag in the parent saying
{% block WHATEVER %}{%endblock%} and in the child template, you need to start with {
%extends 'WHATEVERYOUCALLEDTHEPARENT.tmpl'%}
and then
{%block WHATEVER%} {%endblock%} (note that the names must match)
And then when it loads the child template, it'll look for teh parent, load it, and insert the child template where indicated.
-------------------------------------
About that css stuff. Sure enough, it wasn't finding the css.
I read somewhere about how you can serve static files simply using apache. That makes sense, but I think somehow my django config was interfering with the usual serving of static files from my directory of /var/www.
So, I poked around and noticed that I could just add
(r'^site_media/(?P.*)$', 'django.views.static.serve',{'document_root': './site_media','show_indexes': True}),
to my list of urlpatterns in urls.py. But it wasn't working.
After a bit, i noticed the reason: './site_media' was meant to point to where the static files folder was located on the hard drive. I suppose it's meant to be a relative uri, but it wasn't working. Switched it to a absolute uri and it worked.
--------------------------------------
Starting making another page which was basically identical to my first one
(which just sends out a request and then parses it into a json object and displays it) and got a goofy error saying something about how the dict module had no function named 'push'. Really random. I saw it before htough. And I had the advantage of this page being basically the exact same as the previous one. So I compared them: identical. Then I remembered: sometimes you need to restart apache for the python to get reloaded. So restarted apache and it worked.
-------------------------------------
wanted to be able to log stuff on my server.
SO, I followed the instructions on http://djangosnippets.org/snippets/1731/,
and added 'site_logging' to my list of installed apps, and added the site_logging.py file in the root of my django/mikeServ directory. Then called logging as instructed (even though I would have thought I should have been calling 'site_logging', but just 'logging' did it.)
Viewed the error log using
|tail -f /var/log/apache2/error.log
BOOM BABAY!
--------------------------------