Skip to content

Latest commit

 

History

History
1319 lines (1025 loc) · 37.6 KB

chapter_24_outside_in.asciidoc

File metadata and controls

1319 lines (1025 loc) · 37.6 KB

Finishing "My Lists": Outside-In TDD

Warning, Chapter Recently Updated

🚧 Warning, this Chapter is freshly updated for Django 5 + Python 3.13.

The code listings should have valid syntax, and I’ve been through and sense-checked the chapter text, but a few things might still be off! So let me know what you think of the chapter, via obeythetestinggoat@gmail.com

In this chapter I’d like to talk about a technique called Outside-In TDD. It’s pretty much what we’ve been doing all along. Our "double-loop" TDD process, in which we write the functional test first and then the unit tests, is already a manifestation of outside-in—​we design the system from the outside, and build up our code in layers. Now I’ll make it explicit, and talk about some of the common issues involved.

The Alternative: "Inside-Out"

The alternative to "outside-in" is to work "inside-out", which is the way most people intuitively work before they encounter TDD. After coming up with a design, the natural inclination is sometimes to implement it starting with the innermost, lowest-level components first.

For example, when faced with our current problem, providing users with a "My Lists" page of saved lists, the temptation is to start at the models layer: we probably want to add an "owner" attribute to the List model object, reasoning that an attribute like this is "obviously" going to be required. Once that’s in place, we would modify the more peripheral layers of code, such as views and templates, taking advantage of the new attribute, and then finally add URL routing to point to the new view.

It feels comfortable because it means you’re never working on a bit of code that is dependent on something that hasn’t yet been implemented. Each bit of work on the inside is a solid foundation on which to build the next layer out.

But working inside-out like this also has some weaknesses.

Why Prefer "Outside-In"?

The most obvious problem with inside-out is that it requires us to stray from a TDD workflow. Our functional test’s first failure might be due to missing URL routing, but we decide to ignore that and go off adding attributes to our database model objects instead.

We might have ideas in our head about the new desired behaviour of our inner layers like database models, and often these ideas will be pretty good, but they are actually just speculation about what’s really required, because we haven’t yet built the outer layers that will use them.

One problem that can result is to build inner components that are more general or more capable than we actually need, which is a waste of time, and an added source of complexity for your project. Another common problem is that you create inner components with an API which is convenient for their own internal design, but which later turns out to be inappropriate for the calls your outer layers would like to make…​worse still, you might end up with inner components which, you later realise, don’t actually solve the problem that your outer layers need solved.

In contrast, working outside-in allows you to use each layer to imagine the most convenient API you could want from the layer beneath it. Let’s see it in action.

The FT for "My Lists"

As we work through the following functional test, we start with the most outward-facing (presentation layer), through to the view functions (or "controllers"), and lastly the innermost layers, which in this case will be model code.

We know our create_pre_authenticated_session code works now, so we can just fill out the actual body of our FT to look for a "My Lists" page:

Example 1. src/functional_tests/test_my_lists.py (ch24l001)
from selenium.webdriver.common.by import By
[...]

    def test_logged_in_users_lists_are_saved_as_my_lists(self):
        # Edith is a logged-in user
        self.create_pre_authenticated_session("edith@example.com")

        # She goes to the home page and starts a list
        self.browser.get(self.live_server_url)
        self.add_list_item("Reticulate splines")  # (1)
        self.add_list_item("Immanentize eschaton")
        first_list_url = self.browser.current_url

        # She notices a "My lists" link, for the first time.
        self.browser.find_element(By.LINK_TEXT, "My lists").click()

        # She sees her email is there in the page heading
        self.wait_for(
            lambda: self.assertIn(
                "edith@example.com",
                self.browser.find_element(By.CSS_SELECTOR, "h1").text,
            )
        )

        # And she sees that her list is in there,
        # named according to its first list item
        self.wait_for(
            lambda: self.browser.find_element(By.LINK_TEXT, "Reticulate splines")
        )
        self.browser.find_element(By.LINK_TEXT, "Reticulate splines").click()
        self.wait_for(
            lambda: self.assertEqual(self.browser.current_url, first_list_url)
        )
  1. We’ll define this add_list_item() shortly.

As you can see, we create a list with a couple of items, then we check that this list appears on a new "My Lists" page, and that it’s "named" after the first item in the list.

Let’s validate that it really works by creating a second list, and seeing that appear on the My Lists page as well. The FT continues, and while we’re at it, we check that only logged-in users can see the "My Lists" page:

Example 2. src/functional_tests/test_my_lists.py (ch24l002)
        [...]
        self.wait_for(
            lambda: self.assertEqual(self.browser.current_url, first_list_url)
        )

        # She decides to start another list, just to see
        self.browser.get(self.live_server_url)
        self.add_list_item("Click cows")
        second_list_url = self.browser.current_url

        # Under "my lists", her new list appears
        self.browser.find_element(By.LINK_TEXT, "My lists").click()
        self.wait_for(lambda: self.browser.find_element(By.LINK_TEXT, "Click cows"))
        self.browser.find_element(By.LINK_TEXT, "Click cows").click()
        self.wait_for(
            lambda: self.assertEqual(self.browser.current_url, second_list_url)
        )

        # She logs out.  The "My lists" option disappears
        self.browser.find_element(By.CSS_SELECTOR, "#id_logout").click()
        self.wait_for(
            lambda: self.assertEqual(
                self.browser.find_elements(By.LINK_TEXT, "My lists"),
                [],
            )
        )

Our FT uses a new helper method, add_list_item, which abstracts away entering text into the right input box. We define it in 'base.py':

Example 3. src/functional_tests/base.py (ch24l003)
from selenium.webdriver.common.keys import Keys
[...]

    def add_list_item(self, item_text):
        num_rows = len(self.browser.find_elements(By.CSS_SELECTOR, "#id_list_table tr"))
        self.get_item_input_box().send_keys(item_text)
        self.get_item_input_box().send_keys(Keys.ENTER)
        item_number = num_rows + 1
        self.wait_for_row_in_list_table(f"{item_number}: {item_text}")

And while we’re at it we can use it in a few of the other FTs, like this for example:

Example 4. src/functional_tests/test_layout_and_styling.py (ch24l004-2)
         # She starts a new list and sees the input is nicely
         # centered there too
-        inputbox.send_keys("testing")
-        inputbox.send_keys(Keys.ENTER)
-        self.wait_for_row_in_list_table("1: testing")
+        self.add_list_item("testing")
+

I think it makes the FTs a lot more readable. I made a total of six changes—​see if you agree with me.

A quick run of all FTs, a commit, and then back to the FT we’re working on. The first error should look like this:

$ python src/manage.py test functional_tests.test_my_lists
[...]
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate
element: My lists; [...]

The Outside Layer: Presentation and Templates

The test is currently failing saying that it can’t find a link saying "My Lists". We can address that at the presentation layer, in base.html, in our navigation bar. Here’s the minimal code change:

Example 5. src/lists/templates/base.html (ch24l005)
      <nav class="navbar">
        <div class="container-fluid">
          <a class="navbar-brand" href="/">Superlists</a>
          {% if user.email %}
            <a class="navbar-link" href="#">My lists</a>
            <span class="navbar-text">Logged in as {{ user.email }}</span>
            <form method="POST" action="{% url 'logout' %}">
              [...]

Of course the href="#" means that link doesn’t actually go anywhere, but it does get our FT along to the next failure:

$ python src/manage.py test functional_tests.test_my_lists
[...]
    lambda: self.assertIn(
            ~~~~~~~~~~~~~^
        "edith@example.com",
        ^^^^^^^^^^^^^^^^^^^^
        self.browser.find_element(By.CSS_SELECTOR, "h1").text,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
AssertionError: 'edith@example.com' not found in 'Your To-Do list'

Which is telling us we’re going to have to build a page that at least has the user’s email in its header. Let’s start with the basics—​a URL and a placeholder template for it.

Again, we can go outside-in, starting at the presentation layer with just the URL and nothing else:

Example 6. src/lists/templates/base.html (ch24l006)
  {% if user.email %}
    <a class="navbar-link" href="{% url 'my_lists' user.email %}">My lists</a>

Moving Down One Layer to View Functions (the Controller)

That will cause a template error in the FT:

$ ./src/manage.py test functional_tests.test_my_lists
[...]
Internal Server Error: /
[...]
  File "...goat-book/src/lists/views.py", line 8, in home_page
    return render(request, "home.html", {"form": ItemForm()})
[...]
django.urls.exceptions.NoReverseMatch: Reverse for 'my_lists' not found.
'my_lists' is not a valid view function or pattern name.
[...]
ERROR: test_logged_in_users_lists_are_saved_as_my_lists [...]
[...]
selenium.common.exceptions.NoSuchElementException: [...]

To fix it, we’ll need to start to move from working at the presentation layer, gradually into the controller layer, Django’s view functions.

As always, we start with a test. In this layer, a unit test is the way to go:

Example 7. src/lists/tests/test_views.py (ch24l007)
class MyListsTest(TestCase):
    def test_my_lists_url_renders_my_lists_template(self):
        response = self.client.get("/lists/users/a@b.com/")
        self.assertTemplateUsed(response, "my_lists.html")

That gives:

AssertionError: No templates used to render the response

That’s because the URL doesn’t exist yet, and a 404 has no template. Let’s start our fix in urls.py:

Example 8. src/lists/urls.py (ch24l008)
urlpatterns = [
    path("new", views.new_list, name="new_list"),
    path("<int:list_id>/", views.view_list, name="view_list"),
    path("users/<str:email>/", views.my_lists, name="my_lists"),
]

That gives us a new test failure, which informs us of what we should do. As you can see, it’s pointing us at a views.py, we’re clearly in the controller layer:

    path("users/<str:email>/", views.my_lists, name="my_lists"),
                               ^^^^^^^^^^^^^^
AttributeError: module 'lists.views' has no attribute 'my_lists'

Let’s create a minimal placeholder then:

Example 9. src/lists/views.py (ch24l009)
def my_lists(request, email):
    return render(request, "my_lists.html")

And a minimal template, with no real content except for the header that shows the user’s email address:

Example 10. src/lists/templates/my_lists.html (ch24l010)
{% extends 'base.html' %}

{% block header_text %}{{user.email}}'s Lists{% endblock %}

That gets our unit tests passing.

$ ./src/manage.py test lists
[...]
OK

And hopefully it will address the current error in our FT:

$ python src/manage.py test functional_tests.test_my_lists
[...]
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate
element: Reticulate splines; [...]

Step by step! Sure enough, the FT get a little further. It can now find the email in the <h1>, but it’s now saying that the "My Lists" page doesn’t yet show any lists. It wants them to appear as clickable links, named after the first item.

Another Pass, Outside-In

At each stage, we’re still letting the FT drive what development we do.

Starting again at the outside layer, in the template, we begin to write the template code we’d like to use to get the "My Lists" page to work the way we want it to. As we do so, we start to specify the API we want from the code at the layers below.

A Quick Restructure of Our Template Composition

Let’s take a look at our base template, base.html. It currently has a lot of content that’s specific to editing todo lists, which our "My Lists" page doesn’t need:

Example 11. src/lists/templates/base.html
    <div class="container">

      <nav class="navbar">
        [...]
      </nav>

      {% if messages %}
        [...]
      {% endif %}

      <div class="row justify-content-center p-5 bg-body-tertiary rounded-3">
        <div class="col-lg-6 text-center">
          <h1 class="display-1 mb-4">{% block header_text %}{% endblock %}</h1>

          <form method="POST" action="{% block form_action %}{% endblock %}" >  (1)
            [...]
          </form>
        </div>
      </div>

      <div class="row justify-content-center">
        <div class="col-lg-6">
          {% block table %}  (2)
          {% endblock %}
        </div>
      </div>

    </div>

    <script src="/static/lists.js"></script>  (3)
      [...]
  1. The <form> tag is definitely something we only want on pages where we edit lists. Everything else up to this point is generic enough to be on any page.

  2. Similarly the {% block table %} isn’t something we’d need on the "My Lists" page.

  3. Finally the <script> tag is specific to lists too.

So we’ll want to change things so that base.html is a bit more generic.

Let’s recap: we’ve got three actual pages we want to render:

  1. The home page (where you can enter a first todo item to create a new list)

  2. The "list" page (where you can view an existing list and add to it)

  3. The "my lists" page (which is a list of all your existing lists)

    • The home page and list page both share the "form" elements and the lists.js javascript.

    • But the lists page is the only one that needs to show the full table of list items

    • The "my lists" page doesn’t need anything related to editing or displaying lists.

So we have some things shared between all 3, and some only shared between 1 and 2.

So far we’ve been using inheritance to share the common parts of our templates, but this is a good place to start using composition instead. At the moment we’re saying that "home" is a type of "base" template, but with the "table" section switched off, which is a bit awkward. Let’s not make it even more awkward by saying that "lists" is a "base" template with both the form and the table switched off! It might more sense to say that "home" is a type of base template which "includes" a list form, but no table, and "list" includes both the list form and the list table.

TIP: People often say "prefer composition over inheritance", because inheritance can become hard to reason about as the inheritance hierarchy grows. Composition is more flexible, and often makes more sense. For a lengthy discussion of this topic, see Hynek Schlawack’s definitive article on subclassing in Python.

So, let’s:

  1. Pull out the <form> tag and the lists.js <script> tag into into some blocks we can "include" in our homepage and lists page.

  2. Move the <table> block so it only exists in the lists page.

  3. Take all the lists-specific stuff out of the base.html template, making it into a more generic page with header and a placeholder for generic content:

We’ll use what’s called an include to be able to compose reusable template fragments, when we don’t want to use inheritance.

First let’s pull out the form and the script tag from base.html:

Example 12. src/lists/templates/base.html (ch24l010-1)
@@ -58,43 +58,19 @@
         <div class="col-lg-6 text-center">
           <h1 class="display-1 mb-4">{% block header_text %}{% endblock %}</h1>

-          <form method="POST" action="{% block form_action %}{% endblock %}" >
-            {% csrf_token %}
-            <input
-              id="id_text"
-              name="text"
-              class="form-control
-                     form-control-lg
-                     {% if form.errors %}is-invalid{% endif %}"
-              placeholder="Enter a to-do item"
-              value="{{ form.text.value }}"
-              aria-describedby="id_text_feedback"
-              required
-            />
-            {% if form.errors %}
-              <div id="id_text_feedback" class="invalid-feedback">
-                {{ form.errors.text.0 }}
-              </div>
-            {% endif %}
-          </form>
+          {% block extra_header %}
+          {% endblock %}
+
         </div>
       </div>

-      <div class="row justify-content-center">
-        <div class="col-lg-6">
-          {% block table %}
-          {% endblock %}
-        </div>
-      </div>
+      {% block content %}
+      {% endblock %}

     </div>

-    <script src="/static/lists.js"></script>
-    <script>
-      window.onload = () => {
-        initialize("#id_text");
-      };
-    </script>
+    {% block scripts %}
+    {% endblock %}

   </body>
 </html>

You can see we’ve replaced all the lists-specific stuff with 3 new blocks:

  • extra_header for anything we want to put in the big header section

  • content for the main content of the page

  • scripts for any javascript we want to include.

Let’s paste in the <form> tag into a file at src/lists/templates/form.html:

Example 13. src/lists/templates/form.html (ch24l010-2)
<form method="POST" action="{{ form_action }}" >  (1)
  {% csrf_token %}
  <input
    id="id_text"
    name="text"
    class="form-control
           form-control-lg
           {% if form.errors %}is-invalid{% endif %}"
    placeholder="Enter a to-do item"
    value="{{ form.text.value }}"
    aria-describedby="id_text_feedback"
    required
  />
  {% if form.errors %}
    <div id="id_text_feedback" class="invalid-feedback">
      {{ form.errors.text.0 }}
    </div>
  {% endif %}
</form>
  1. This is the only change, we’ve replaced the {% block form_action %} with {{ form_action }}.

Let’s paste the scripts tags verbatim into a new file at src/lists/templates/scripts.html:

Example 14. src/lists/templates/scripts.html (ch24l010-3)
<script src="/static/lists.js"></script>

<script>
  window.onload = () => {
    initialize("#id_text");
  };
</script>

Now let’s look at how to use the include, and how the form_action change plays out, in the changes to home.html:

Example 15. src/lists/templates/home.html (ch24l010-4)
{% extends 'base.html' %}

{% block header_text %}Start a new To-Do list{% endblock %}

{% block extra_header %}
  {% url 'new_list' as form_action %}  (1)
  {% include "form.html" with form=form form_action=form_action %}  (2)
{% endblock %}

{% block scripts %}  (3)
  {% include "scripts.html" %}
{% endblock %}
  1. The {% url …​ as %} syntax lets us define a template variable in-line

  2. Then we use {% include …​ with key=value…​ %} to pull in the contents of the form.html template, with the appropriate context variables passed in—​a bit like calling a function.

  3. The scripts block is just a straightforward include with no variables.

Now let’s see it in list.html:

Example 16. src/lists/templates/list.html (ch24l010-5)
@@ -2,12 +2,24 @@

 {% block header_text %}Your To-Do list{% endblock %}

-{% block form_action %}{% url 'view_list' list.id %}{% endblock %}

-{% block table %}
+{% block extra_header %}  (1)
+  {% url 'view_list' list.id as form_action %}
+  {% include "form.html" with form=form form_action=form_action %}
+{% endblock %}
+
+{% block content %}  (2)
+<div class="row justify-content-center">
+  <div class="col-lg-6">
     <table class="table" id="id_list_table">
       {% for item in list.item_set.all %}
         <tr><td>{{ forloop.counter }}: {{ item.text }}</td></tr>
       {% endfor %}
     </table>
+  </div>
+</div>
+{% endblock %}
+
+{% block scripts %}  (3)
+  {% include "scripts.html" %}
 {% endblock %}
  1. The block table becomes an extra_header block, and we use the include to pull in the form.

  2. The block table becomes a content block, with all the html we need for our table.

  3. And the scripts block is the same as the one from home.html.

We can have a little click around our site, and then a little re-run of all our FTs to make sure we haven’t broken anything, and then commit.

$ ./src/manage.py test functional_tests
[...]
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate
element: Reticulate splines; [...]
[...]
Ran 8 tests in X.Xs

FAILED (errors=1)

8 tests with 1 failure, the same one we had before, we haven’t broken anything. Hooray!

$ git add src/lists/templates
$ git commit -m "refactor templates to use composition/includes"

Now let’s get back to our outside-in process, and working in our template to drive out the requirements for our views layer:

Designing Our API Using the Template

So, in my_lists.html we can now work in the content block:

Example 17. src/lists/templates/my_lists.html (ch24l010-6)
{% extends 'base.html' %}

{% block header_text %}{{user.email}}'s Lists{% endblock %}

{% block content %}
  <h2>{{ owner.email }}'s lists</h2>  (1)
  <ul>
    {% for list in owner.lists.all %}  (2)
      <li><a href="{{ list.get_absolute_url }}">{{ list.name }}</a></li>  (3)
    {% endfor %}
  </ul>
{% endblock %}

We’ve made several design decisions in this template which are going to filter their way down through the code:

  1. We want a variable called owner to represent the user in our template.

  2. We want to be able to iterate through the lists created by the user using owner.lists.all (I happen to know how to make this work with the Django ORM).

  3. We want to use list.name to print out the "name" of the list, which is currently specified as the text of its first element.

Note
Outside-In TDD is sometimes called "programming by wishful thinking",[1] and you can see why. We start writing code at the higher levels based on what we wish we had at the lower levels, even though it doesn’t exist yet…​ A bit like when we write test for code that doesn’t exist yet!

We can rerun our FTs, to check that we didn’t break anything, and to see whether we’ve got any further:

$ python src/manage.py test functional_tests
[...]
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate
element: Reticulate splines; [...]

 ---------------------------------------------------------------------
Ran 8 tests in 77.613s

FAILED (errors=1)

Well, no further, but at least we didn’t break anything. Time for a commit:

$ git add src/lists
$ git diff --staged
$ git commit -m "url, placeholder view, and first-cut templates for my_lists"

Moving Down to the Next Layer: What the View Passes to the Template

Now our views layer needs to respond to the requirements we’ve laid out in the template layer, by giving it the objects it needs. In this case, the list owner:

Example 18. src/lists/tests/test_views.py (ch24l011)
from django.contrib.auth import get_user_model

User = get_user_model()
[...]


class MyListsTest(TestCase):
    def test_my_lists_url_renders_my_lists_template(self):
        [...]

    def test_passes_correct_owner_to_template(self):
        User.objects.create(email="wrong@owner.com")
        correct_user = User.objects.create(email="a@b.com")
        response = self.client.get("/lists/users/a@b.com/")
        self.assertEqual(response.context["owner"], correct_user)

Gives:

KeyError: 'owner'

So:

Example 19. src/lists/views.py (ch24l012)
from django.contrib.auth import get_user_model

User = get_user_model()
[...]


def my_lists(request, email):
    owner = User.objects.get(email=email)
    return render(request, "my_lists.html", {"owner": owner})

That gets our new test passing, but we’ll also see an error from the previous test. We just need to add a user for it as well:

Example 20. src/lists/tests/test_views.py (ch24l013)
    def test_my_lists_url_renders_my_lists_template(self):
        User.objects.create(email="a@b.com")
        [...]

And we get to an OK:

OK

The Next "Requirement" from the Views Layer: New Lists Should Record Owner

Before we move down to the model layer, there’s another part of the code at the views layer that will need to use our model: we need some way for newly created lists to be assigned to an owner, if the current user is logged in to the site.

Here’s a first crack at writing the test:

Example 21. src/lists/tests/test_views.py (ch24l014)
class NewListTest(TestCase):
    [...]

    def test_list_owner_is_saved_if_user_is_authenticated(self):
        user = User.objects.create(email="a@b.com")
        self.client.force_login(user)  #(1)
        self.client.post("/lists/new", data={"text": "new item"})
        new_list = List.objects.get()
        self.assertEqual(new_list.owner, user)
  1. force_login() is the way you get the test client to make requests with a logged-in user.

The test fails as follows:

AttributeError: 'List' object has no attribute 'owner'

To fix this, we can try writing code like this:

Example 22. src/lists/views.py (ch24l015)
def new_list(request):
    form = ItemForm(data=request.POST)
    if form.is_valid():
        nulist = List.objects.create()
        nulist.owner = request.user
        nulist.save()
        form.save(for_list=nulist)
        return redirect(nulist)
    else:
        return render(request, "home.html", {"form": form})

But it won’t actually work, because we don’t know how to save a list owner yet:

    self.assertEqual(new_list.owner, user)
                     ^^^^^^^^^^^^^^
AttributeError: 'List' object has no attribute 'owner'

A Decision Point: Whether to Proceed to the Next Layer with a Failing Test

In order to get this test passing, as it’s written now, we have to move down to the model layer. However, it means doing more work with a failing test, which is not ideal.

The alternative is to rewrite the test to make it more isolated from the level below, using mocks.

On the one hand, it’s a lot more effort to use mocks, and it can lead to tests that are harder to read. On the other hand, advocates of what’s known as "London School" TDD are very keen on the approach. Read more in [appendix_purist_unit_tests].

For now we’ll accept the tradeoff, moving down one layer with failing tests, but avoiding the extra mocks.

Let’s do a commit, and then tag the commit as a way of remembering our position for that appendix:

$ git commit -am "new_list view tries to assign owner but cant"
$ git tag revisit_this_point_with_isolated_tests

Moving Down to the Model Layer

Our outside-in design has driven out two requirements for the model layer: we want to be able to assign an owner to a list using the attribute .owner, and we want to be able to access the list’s owner with the API owner.lists.all().

Let’s write a test for that:

Example 23. src/lists/tests/test_models.py (ch24l018)
from django.contrib.auth import get_user_model
[...]

User = get_user_model()
[...]


class ListModelTest(TestCase):
    def test_get_absolute_url(self):
        [...]

    def test_lists_can_have_owners(self):
        user = User.objects.create(email="a@b.com")
        mylist = List.objects.create(owner=user)
        self.assertIn(mylist, user.lists.all())

And that gives us a new unit test failure:

    mylist = List.objects.create(owner=user)
    [...]
TypeError: List() got unexpected keyword arguments: 'owner'

The naive implementation would be this:

from django.conf import settings
[...]

class List(models.Model):
    owner = models.ForeignKey(settings.AUTH_USER_MODEL)

But we want to make sure the list owner is optional. Explicit is better than implicit, and tests are documentation, so let’s have a test for that too:

Example 24. src/lists/tests/test_models.py (ch24l020)
    def test_list_owner_is_optional(self):
        List.objects.create()  # should not raise

The correct implementation is this:

Example 25. src/lists/models.py (ch24l021)
from django.conf import settings
[...]

class List(models.Model):
    owner = models.ForeignKey(
        settings.AUTH_USER_MODEL,
        related_name="lists",
        blank=True,
        null=True,
        on_delete=models.CASCADE,
    )

    def get_absolute_url(self):
        return reverse("view_list", args=[self.id])

Now running the tests gives the usual database error:

    return super().execute(query, params)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
django.db.utils.OperationalError: table lists_list has no column named owner_id

Because we need to make some migrations:

$ python src/manage.py makemigrations
Migrations for 'lists':
  src/lists/migrations/0007_list_owner.py
    + Add field owner to list

We’re almost there; a couple more failures in some of our old tests:

ERROR: test_can_save_a_POST_request
[...]
ValueError: Cannot assign "<SimpleLazyObject:
<django.contrib.auth.models.AnonymousUser object at 0x1069852e>>": "List.owner" must
be a "User" instance.
[...]

ERROR: test_redirects_after_POST
[...]
ValueError: Cannot assign "<SimpleLazyObject:
<django.contrib.auth.models.AnonymousUser object at 0x106a1b440>>": "List.owner" must
be a "User" instance.

We’re moving back up to the views layer now, just doing a little tidying up. Notice that these are in the existing test for the new_list view, when we haven’t got a logged-in user.

The tests are reminding us to think of this use case too: we should only save the list owner when the user is actually logged in. The .is_authenticated attribute we came across in [chapter_19_spiking_custom_auth] comes in useful now (when they’re not logged in, Django represents users using a class called AnonymousUser, whose .is_authenticated is always False):

Example 26. src/lists/views.py (ch24l023)
    if form.is_valid():
        nulist = List.objects.create()
        if request.user.is_authenticated:
            nulist.owner = request.user
            nulist.save()
        form.save(for_list=nulist)
        return redirect(nulist)
        [...]

And that gets us passing!

$ python src/manage.py test lists
[...]

Ran 38 tests in 0.237s

OK

This is a good time for a commit:

$ git add src/lists
$ git commit -m "lists can have owners, which are saved on creation."

Final Step: Feeding Through the .name API from the Template

The last thing our outside-in design wanted came from the templates, which wanted to be able to access a list "name" based on the text of its first item:

Example 27. src/lists/tests/test_models.py (ch24l024)
    def test_list_name_is_first_item_text(self):
        list_ = List.objects.create()
        Item.objects.create(list=list_, text="first item")
        Item.objects.create(list=list_, text="second item")
        self.assertEqual(list_.name, "first item")
Example 28. src/lists/models.py (ch24l025)
    @property
    def name(self):
        return self.item_set.first().text

And that, believe it or not, actually gets us a passing test, and a working "My Lists" page (The "My Lists" page, in all its glory (and proof I did test on Windows))!

$ python src/manage.py test functional_tests
[...]
Ran 8 tests in 93.819s

OK
Screenshot of new My Lists page
Figure 1. The "My Lists" page, in all its glory (and proof I did test on Windows)
The @property Decorator in Python

If you haven’t seen it before, the @property decorator transforms a method on a class to make it appear to the outside world like an attribute.

This is a powerful feature of the language, because it makes it easy to implement "duck typing", to change the implementation of a property without changing the interface of the class. In other words, if we decide to change .name into being a "real" attribute on the model, which is stored as text in the database, then we will be able to do so entirely transparently—​as far as the rest of our code is concerned, they will still be able to just access .name and get the list name, without needing to know about the implementation. Raymond Hettinger gave a great, beginner-friendly talk on this topic at Pycon a few years ago, which I enthusiastically recommend (it covers about a million good practices for Pythonic class design besides).

Of course, in the Django template language, .name would still call the method even if it didn’t have @property, but that’s a particularity of Django, and doesn’t apply to Python in general…​

But we know we cheated to get there. The Testing Goat is eyeing us suspiciously. We left a test failing at one layer while we implemented its dependencies at the lower layer. Let’s see how things would play out if we were to use better test isolation…​

Outside-In TDD
Outside-In TDD

A methodology for building code, driven by tests, which proceeds by starting from the "outside" layers (presentation, GUI), and moving "inwards" step by step, via view/controller layers, down towards the model layer. The idea is to drive the design of your code from how it will be used, rather than trying to anticipate requirements from the bottom up.

Programming by wishful thinking

The outside-in process is sometimes called "programming by wishful thinking". Actually, any kind of TDD involves some wishful thinking. We’re always writing tests for things that don’t exist yet.

The pitfalls of outside-in

Outside-in isn’t a silver bullet. It encourages us to focus on things that are immediately visible to the user, but it won’t automatically remind us to write other critical tests that are less user-visible—​things like security, for example. You’ll need to remember them yourself.


1. The phrase "programming by wishful thinking" was first popularised by the amazing, mind-expanding textbook SICP, which I cannot recommend highly enough.