Django Bootstrap Toolkit

For this project's web interface, we chose to use Twitter Bootstrap. This way, a lot of design things we would have otherwise had to build manually, we can draw upon a pre-existing pack of styles. There were a few unique issues we ran across as we got this to work, however. For starters, Django will not integrate straight-up Twitter Bootstrap because it doesn't know how. We need a way to use Bootstrap THROUGH Django. After a little bit of Google searching, I ran across a Youtube video tutorial that someone was, among other things, doing exactly this: installing Bootstrap through Django.
The link is as follows: http://youtu.be/utR1KtRFvxg
It was actually quite easy in the long run. Assuming you already have pip3 for python3, you can just run this command:

sudo pip3 install django-bootstrap-toolkit

This will install the django-bootstrap-toolkit for your python3. Once this is complete, you will need to modify your settings.py. The section that reads:

INSTALLED_APPS = (
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
)

or something like that, just add the following to the end of the list:  'bootstrap_toolkit',

so it would look like this or more. Just make sure you have the bootstrap_toolkit.

INSTALLED_APPS = (
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'bootstrap_toolkit',
)

Django is now using the bootstrap_toolkit.

Then comes the issue of actually using it. As you could imagine, with the Django integration, you can't always just copy a template from http://getbootstrap.com because, again, Django doesn't know what you're doing if you do it wrong. To start, I couldn't just download the .zip from bootstrap's site. I had to load external css pages because Django didn't know where I was actually putting those files so it didn't use them. If I use external style sheets, they will always work regardless of whether Django know's it's there or not. I did the following:

<meta charset=”utf-8”>
<!-- Latest compiled and minified CSS -->
<link rel="stylesheet" href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css">

<!-- Optional theme -->
<link rel="stylesheet" href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap-theme.min.css">

This loads the css pages. I put this within the header tag. Then, for the javascript needed for bootstrap, I did the following:

<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.0/jquery.min.js"></script>
    <!-- Latest compiled and minified JavaScript -->
<script src="//netdna.bootstrapcdn.com/bootstrap/3.1.1/js/bootstrap.min.js"></script>

These lines go right before the close of the body tag because I read that this makes the page load a little faster.

You can now start copying and pasting templates from the twitter bootstrap website to see how they work with your project. I added a navbar to every page by putting it in the base.html template. This ended up covering content. Should you choose to take the same route, make sure to add something like this somewhere in your document:

<style type="text/css">
body{padding-top: 60px;}
</style>

this pushes everything down a bit. Finally, to add the Django dynamic content, put the following within your body tag:

<div class="container">
{% block content %}
{% endblock %}
</div>

put this after any header navbar and your content will appear under the navbar. The {% block content %} and {% endblock %} tags tell Django to put the dynamic content here.

At this point, you experiment. Try adding a visual feature and integrate it in the templates. If it breaks, either fix it or try something different.

Arik

Standing Meeting 5: Report from the First Split

This week was the first time that our team tried assigning specific tasks to specific people, let's see how we did.

• Arik:
• Make the website look pretty. All that we have so far is black text displayed on a white background. Arik plans on looking into how Twitter Bootstrap can be integrated into Django.
• Zach:
• Complete tests to simulate Amber's success story. This will be accomplished by checking to make sure the views that are returned are the correct views for a logged in user.
• Karl: 
• Re-organize code so that authentication files are contained in one location. Implement the Light model to the project.
• Entire Group:
• Write tests for the basic model structure that will be implemented for this week.
• Further solidify the UML diagram for the project.

Arik's success of integrating twitter bootstrap into our project is a huge confidence boost. It makes our website look like it wasn't built by a fourth grader. Check out our new homepage!

Zach made significant progress towards attaining his goal, but difficulties with the svn repository due to power outages prevented him from being able to complete his goal for the week. 

Karl successfully reorganized the authentication files after quite a bit of difficulty regarding URLs and templates. A light model has been successfully added so that an administrator can add lights to a specific user.

Our lack of tests added for the models to be implemented is due in part to our unfamiliarity with the test first coding practice and also in part due to ambiguous UML diagrams. Since we aren't familiar with how to write tests and exactly what tests to write, it is difficult to complete the task.

There really isn't much excuse for the lack of a further solidified UML, this will be our first priority for next week, hopefully allowing other pieces of the project to be more easily understood and implemented.

These are our goals for next week (or whenever, since we will be going on Spring Break next week):

• Arik:
• Continue to update new pages which we develop to be wrapped in the beautiful environment that is twitter bootstrap.
• Develop the House model for the management app.
• Implement tests for the house model.
• Adding a house to the database.
• Adding a room to the house.
• Removing a room from the house.
• Accessing a house by the user.
• Zach
• Complete tests to simulate Amber's success story. This will be accomplished by checking to make sure the views that are returned are the correct views for a logged in user.
• Develop the Room model for the management app.
• Implement tests for the room model.
• Adding a room to the database.
• Adding a light to a room.
• Removing a light from the room.
• Karl
• Solidify the UML diagrams for the project. Make these in the Violet UML editor.
• Provide users with a view to see what lights are in their house, not necessarily organized by house or room.
• Allow users to change the state of a light.
• Whole group
• Start programming on the Pis, install all necessary software on the devices so that they can function as our "houses".
• Implement Napoleon's success story.

Napoleon is a registered user of the project HAM website. He wants to see how many lights are currently on in his house. He logs into the website and is immediately greeted by a display which indicates how many lights are controlled by the system in his house and whether those lights are off or on.

The Registration Page

Wilbur's success story was implementing a view that would allow a user to create an account on our website. Like a large majority of the things that we do with our website, the way to implement this came from a very helpful post on Stack Overflow.

Unfortunately, while this provided all the details for how to log the user in behind the scenes, it doesn't provide the method of allowing the user to input their data. Thankfully, this webpage provided a template that can be used for logging a user in. Using this template with the code from the previous post should work right?

Wrong! It was at this point that I was introduced to one of Django's security features, csrf tokens. Cross Site Request Forgery Tokens are used to prevent malicious websites from accessing our websites data. Sounds like a good idea to me, as long as I don't have to spend valuable hours of sleep figuring out how to make it work with our software engineering project. The fix was simple enough, I just had to add one line of code:

{% extends "base.html" %}
{% block title %}Create an account{% endblock %} 
{% block content %} 
<h1>Create an account</h1> 
<form action="" method="post"> 
    {% csrf_token %} <--------------This line needs to be added.
    {{ form.as_p }} 
    <input type="submit" value="Create the account"> 
</form> 
{% endblock %}

Now I don't know if my site will be more protected because I added this one line of code, but I do know that it lets me experience success logging in, and that's what's most important to me right now!

Logging in Unit Tests

This past week I've been working on implementing Amber's success story into 'unit test format.'  We already had a test making sure that a user is added to the database, but we thought that actually logging in was something different.  I had little to no experience with unit tests, and the environment so my first few attempts were spent figuring out how things worked.

Eventually I came across a hidden section in the django tutorial using Clients.  This is perfect for what I was looking for, as I can use the client to 'post' their username and password to the login page.  I created two tests, one for a successful login (correct username and correct password), and one for an unsuccessful login (correct username, incorrect password).  The way I was checking to make sure it was working (for the successful login) is listed below:

1. Create test user (Username = Amber, Password = password)
2. Create client
3. Client posts <Username = Amber, Password = password> at '/login/'
4. Save the response code from the post.
5. Check to make sure that the HTTP response code was equal to 200 (the OK response code).

I did the same process with the incorrect login information, and checked to see if the response code WASN'T equal to 200.  The successful login worked, and the unsuccessful login test failed.

After I spent more time messing around with the tests, I discovered the problem.  After a login attempt, successful or otherwise, the system redirects the user towards the correct page (profile if logged in, and login page if not).  Well the system does a GET request to grab that correct page, and the HTTP request always sends back a 200.  That means no matter what, both tests were always returning a 200.  The HTTP request I was saving was bypassing the POST request and going straight to the GET request.  I am not sure why this is, but it is obviously a problem.  

My next possible solution is to have the client login, then check what view the user is currently on.  For the successful login I will assert that [location] (which is saved along with the HTTP code) is equal to /accounts/profile.  While with the unsuccessful login I will assert that [location] is equal to /login/.

I haven't gotten time to attempt to implement this new strategy, as we were unable to pull our project from our SVN repository on Dijkstra.  I will edit this post once I have implemented it.

Standing Meeting 4: Breaking Out

Let's start the review of our standing meeting how we always do, what is our progress from last week?

• Create a subversion repository for our project.
• Implement tests to simulate Amber's success story.
• Develop a mock-up of what we expect the user interface might look like.
• Implement Wilbur's success story.
• Make a preliminary structure of the models that we plan to use in our project.

The past week was very successful, as evident from the number of items crossed off our list. I think that we were even more successful than the list indicates however, because we figured out how we can work more effectively as a team. 

Creating a subversion repository for our project turned out to be an easy goal for this week because we were required to do it in class just after we set the goal for ourselves in our standing meeting. After a little confusion about what the project would be named and who would upload it, we managed to have a version stored that everyone was able to download and simulate the success stories that we had already put into place.

Amber's success story made some initial steps toward completion. We have some tests that create a user and check that they are in the database, but we need to do a little bit more research into how to simulate the user navigating to the website and signing in. Zach will be in charge of completing this goal for next week. 

We developed a mock-up of what the user interface might look like, as well as making a rough UML-esque diagram of the classes and method we expect to use in our project. I say UML-esque because I am fairly certain that we didn't follow all of the conventions of the UML. Despite this, I think it was a very useful exercise. In fact, I think that these two activities were probably the most significant achievements of this week. By outlining how we expect the program might be organized, we have a more unified idea of what the project will look like. 

We also implemented Wilbur's success story. You can expect a separate blog post describing that in the near future.

Before our standing meeting this week, we had another group meeting that was very significant. As we have been working on the project, we have just had one major success story that we tried to implement and we had everyone in the group try to work towards that goal independently. We realized that this doesn't work very well because there isn't a lot of motivation to work on the project. Each group member can independently hope that the other group members will accomplish something. From this point forward, we've decided to give each group member a specific task to accomplish so that everyone has an individual goal to work towards. Even though we have these individual goals, we also determined that it will be easier to achieve them if we gather as a project team to work on them. We plan on having regular meeting times scheduled from this point forward.

So with these points in mind, here are our goals for next week:

• Arik:
• Make the website look pretty. All that we have so far is black text displayed on a white background. Arik plans on looking into how Twitter Bootstrap can be integrated into Django.
• Zach:
• Complete tests to simulate Amber's success story. This will be accomplished by checking to make sure the views that are returned are the correct views for a logged in user.
• Karl: 
• Re-organize code so that authentication files are contained in one location. Implement the Light model to the project.
• Entire Group:
• Write tests for the basic model structure that will be implemented for this week.
• Further solidify the UML diagram for the project.

Here's a sneak peak at the UML we started to develop.

Templates

What's a programmer's favorite part of programming? The finished product? The process of learning a new skill to implement some functionality? Building new worlds to explore? Sure these are all reasons why people might enjoy programming, but I'm going to explain the strange reason why I enjoy programming.

I was working on our website, and was trying to change our view from delivering a simple HTTPResponse() text string to delivering a rendered template file. In doing this, I tried to follow the Django tutorial, but I kept on receiving an error that the template file could not be located. Googling for solutions to the problem just returned results about making sure that template files for specific models are stored with the models. This isn't what I was trying to accomplish. I just wanted a general view for the website. The error explained that it searched for templates in: 

.../projectHAM/templates/site/homepage.html

 but couldn't find the resource specified.

"Umm, excuse me?" I said, "I clearly have that template located in the directory. Just look at my directory structure!"

Clearly there was something wrong with Django, because surely I couldn't have been doing anything wrong. How can I correct an error that the template isn't in the right space when it clearly is located in the right spot. Frustrated and hungry because it was 6:45 pm and I hadn't eaten yet, I left the obstinate computer in a huff.

After coming back from dinner, I sat down to try and make a deal with the computer. Maybe it would be willing to work with me after it had time to cool down a little. It wasn't long until I was able to fix the problem. Can you guess what my error was?

I mentioned earlier that the error said that the template should be located at 

.../projectHAM/templates/site/homepage.html

which is the way I read it when I was trying to solve the problem. What the error actually said was

.../workspace/projectHAM/templates/site/homepage.html

while I had my templates located in

 .../projectHAM/projectHAM/templates/site/homepage.html

I had spent probably 45 minutes trying to figure out what was wrong with Django when I was the one that placed the templates folder in the wrong directory. After this glorious, "Aha!" moment I moved the template directory to the proper place, reloaded the page, and saw the error screen no more.

This enormous struggle that I had that ended in that wonderful moment when you I realized what I had done wrong and exactly what I needed to do to fix it is why programming is so much fun for me. Those small moments when your world is consumed by a problem and then you finally fix it and everything feels right.

With that said, there are two lessons to take away from this:

1. If you spend more than 30 minutes trying to fix an error and aren't sure what to do just drop it, walk away and come back to the problem later. A fresh perspective is one of the most useful tools for solving a problem.
2. Read the whole directory structure, of where template files are supposed to be located. It will save you approximately 45 minutes of struggling with what seems to be an impossible problem.

Now that our website uses templates for returning views, we can do some pretty cool features, like identifying our users by name when they log in. Here is our new "congrats, you're logged in" screen.

Unit Test Exceptions in Python

One of the goals for the next meeting was to develop tests for logging a user in to the system. This is something that is to me personally as a software developer, so I was grateful that the Django tutorial covered how to run tests in Python/Django. I did run into some difficulty though, when it came to trying to retrieve a user that was not in the database. Trying to do this would raise an exception during execution of the program.The Django framework has a method for expected exceptions called assertRaises(), but when I tried to use this, I kept getting an exception during run-time.

I thought to myself, "What's the point of this silly method? It's supposed to detect when an exception is raised, but it doesn't catch the exception before it is thrown." Here is how I was calling the assertRaises() method.

self.assertRaises(User.DoesNotExist, User.objects.get_by_natural_key('fyodor'))

Can you guess why I was still getting an exception? I'll give you a hint, it has something to do with how function argument are passed in the stack frame. Still stumped? I'll show you the solution, and maybe that will give you an idea.

with self.assertRaises(User.DoesNotExist):
    User.objects.get_by_natural_key('fyodor') 

The exception was still being raised because when function had arguments, they are placed on the stack before the function is called. This means that User.objects.get_by_natural_key() was being called before the program called the assertRaises() function was called. The program had no idea that the exception was expected at that point, so the test kept on failing.

At least, that's how I made sense of the scenario.

Standing Meeting 3: Successes and Failures

As always, we begin our standing meeting by reviewing our goals from last week.

• Create a simple log in page for the website. 
• Create a subversion repository for our project.
• Implement tests to simulate Amber's success story.
• Allow Amber to experience success in her user story.
• Develop a mock-up of what we expect the user interface might look like.

This week, we for the first time experienced both success and failure in our goals. We were able to make a website for our users to log in to, but failed to build the framework on the back end to build our project. I think that we are still going through the stages of becoming familiar with Python and Django, which is keeping us from being able to look to the future of our project. As we become more aware of our tools and what we can do with them, we will be able to visualize how our project might form so that we can begin building that road. To make a very poor analogy, we are chefs who are cooking a dish from a foreign country. We have cooking (programming) skills already, but we are most familiar with the foods (programming language, backend infrastructure) of our own country (Java, C). Until we've had the chance to experiment with the foods from the foreign country (Python, Django, Web application programming, unit tests, subversion repositories) we won't know how to craft them into an expert dish.

With that said, we have to accept that some of our goals might be unattainable with our present skills, but they might be more accessible later on. We added to our incomplete goals a few new goals for this week.

• Create a subversion repository for our project.
• Implement tests to simulate Amber's success story.
• Develop a mock-up of what we expect the user interface might look like.
• Implement Wilbur's success story.
• Make a preliminary structure of the models that we plan to use in our project.

What's that you say? What is Wilbur's success story? Well I'm glad you asked.

Wilbur is interested in this new craze going around his neighborhood, he's heard something about being able to turn lights on in his house without actually touching the light switch. His friend, Amber, says that he can sign up for an account with the HAM website and request that a technician visits his house to set up the system. Wilbur isn't sure about having a stranger coming into his house to fiddle with electronics, but he figures it couldn't hurt to sign up for a website. He navigates to the HAM website and creates a new account. He then logs in to the website.

While we currently aren't sure if this would be something that we would allow at our company (maybe users would have to install the system before they can have an account on the website) we hope that it will be good to experiment a little bit with the food and learn something in the process.

Amber's Success

After a long period of struggling, Amber has finally experienced success with her user story! Amber was able to access the website and log in using her name and a password. Amber is slightly confused by the website though, all that shows up is some small 12pt font in the upper left corner that lets her know that she has accessed the website. She then has to add "/login" to the address bar to bring up the log in screen so that she can enter her credentials. After logging in, a familiar mostly blank screen appears with some different text this time that says "You are now logged in!" Even more confusing, in order to log out, she has to navigate to a "/logout" page. Not the most useful website in the world, but she feels secure knowing that she has an account with this website.

Finding out how to get a standard user to log in with Django's framework proved to be a fairly complex process. After wading through seemingly endless looping links in the Django documentation, I finally stumbled on this page that seemed to be just what I needed. I found that looking at the "login_required" section was particularly helpful. Adding this redirected me to a predefined URL which should host the log in page. Unfortunately, there was an error with displaying this log in page. I was probably missing some obvious piece of code, but it seemed to me that the django directory was missing some template files. By searching in the directories for this log in template, I was able to come across another log in template for the admin half of the site. I pasted this template into the directory that the log in page was looking for the template and I had an instant (though incorrectly labelled) log in page. Using my knowledge from the Django tutorial, I put together the simple text pages that greet the user to the website and notify them that they have logged in.

I hope to come up with a more elegant solution to this log in problem in the future, but for the moment, it feels so good to have completed this first step.

Django Woes: "You mean I don't have to write code?"

While I was going through the tutorial for Django, I have had some fairly emotional experiences (considering that all I am doing is writing computer code). The first one was when I was accessing the admin portion of the site that I had just effortlessly created. I then recreated the view of the polls so that instead of just listing the text of each of the entries in the database, it also displayed the date that they were published and whether or not they were published recently. I then edited the code so that Django was aware that the was_published_recently() method returned a boolean value. After doing this, little green check marks appeared where True was earlier displayed. This simple aesthetic change was so unexpected and pleasant, I may or may not have raised my fist in the air and screamed a little bit.

The second emotional experience I had was in tutorial 4. We wrote some pretty generic views in tutorial 3, and then we replaced a good chunk of the code in tutorial 4 with generic code that took up a lot less space. I should have been happy that there was a much easier way to do the code that I had just written, but instead I was angry that I had to erase the code that I had so carefully crafted earlier. This is something that I should learn to overcome as an extreme programmer though, because there might often be times when it will be better for me to get rid of code and write new code than stick with older code.

A final note on the Django tutorial. Since it went through the process of logging in with an admin account, I thought it would also go through the process of how you could implement user accounts. Unfortunately it didn't, leaving our project in a precarious place. There seems to already be a useful built in user account system, but I worry that we won't be able to implement it correctly and cause problems later in the development process.