A planet of blogs from our members...

Caktus Group—Django Depends on You: A Takeaway from DjangoCon

DjangoCon 2018 attracted attendees from around the world, including myself and several other Cakti (check out our DjangoCon recap post). Having attended a number of DjangoCons in the past, I looked forward to reconnecting with old colleagues and friends within the community, learning new things about our favorite framework, and exploring San Diego.

While it was a privilege to attend DjangoCon in person, you can experience it remotely. Thanks to technology and the motivated organizers, you can view a lot of the talks online. For that, I am thankful to the DjangoCon organizers, sponsors, and staff that put in the time and energy to ensure that these talks are readily available for viewing on YouTube.

Learn How to Give Back to the Django Framework

While I listened to a lot of fascinating talks, there was one that stood out and was the most impactful to me. I also think it is relevant and important for the whole Django community. If you have not seen it, I encourage you to watch and rewatch Carlton Gibson’s “Your web framework needs you!". Carlton was named a Django Fellow in January of 2018 and provides a unique perspective on the state of Django as an open source software project, from the day-to-day management, to the (lack of) diversity amongst the primary contributors, to the ways that people can contribute at the code and documentation levels.

This talk resonated with me because I have worked with open source software my entire career. It has enabled me to bootstrap and build elegant solutions with minimal resources. Django and its ilk have afforded me opportunities to travel the globe and engage with amazing people. However, in over 15 years of experience, my contributions back to the software and communities that have served me well have been nominal in comparison to the benefits I have received. But I came away from the talk highly motivated to contribute more, and am eager to get that ball rolling.

Carlton says in his talk, “we have an opportunity to build the future of Django here.” He’s right, our web framework needs us, and via his talk you will discover how to get involved in the process, as well as what improvements are being made to simplify onboarding. I agree with Carlton, and believe it’s imperative to widen the net of contributors by creating multiple avenues for contributions that are easily accessible and well supported. Contributions are key to ensuring a sound future for the Django framework. Whether it’s improving documentation, increasing test coverage, fixing bugs, building new features, or some other aspect that piques your interest, be sure to do your part for your framework. The time that I am able to put toward contributing to open source software has always supplied an exponential return, so give it a try yourself!

Watch the talk to see how you can contribute to the Django framework.

Caktus Group—DjangoCon 2018 Recap

Above: Hundreds of happy Djangonauts at DjangoCon 2018. (Photo by Bartek Pawlik.)

That’s it, folks — another DjangoCon in the books! Caktus was thrilled to sponsor and attend this fantastic gathering of Djangonauts for the ninth year running. This year’s conference ran from October 14 - 19, in sunny San Diego. ☀️

Our talented Caktus contractor Erin Mullaney was a core member of this year’s DjangoCon organizing team, plus five more Cakti joined as participants: CTO Colin Copeland, technical manager Karen Tracey, sales engineer David Ray, CBDO Ian Huckabee, and myself, account exec Tim Scales.

What a Crowd!

At Caktus we love coding with Django, but what makes Django particularly special is the remarkable community behind it. From the inclusive code of conduct to the friendly smiles in the hallways, DjangoCon is a welcoming event and a great opportunity to meet and learn from amazing people. With over 300 Django experts and enthusiasts attending from all over the world, we loved catching up with old friends and making new ones.

What a Lineup!

DjangoCon is three full days of impressive and inspiring sessions from a diverse lineup of presenters. Between the five Cakti there, we managed to attend almost every one of the presentations.

We particularly enjoyed Anna Makarudze’s keynote address about her journey with coding, Russell Keith-Magee’s hilarious talk about tackling time zone complexity, and Tom Dyson’s interactive presentation about Django and Machine Learning. (Videos of the talks should be posted soon by DjangoCon.)

What a Game!

Thanks to the 30+ Djangonauts who joined us for the Caktus Mini Golf Outing on Tuesday, October 16! Seven teams putted their way through the challenging course at Belmont Park, talking Django and showing off their mini golf skills. We had fun meeting new friends and playing a round during the beautiful San Diego evening.

Thanks to all the organizers, volunteers, and fellow sponsors who made DjangoCon 2018 a big success. We look forward to seeing you again next year!

Caktus Group—Filtering and Pagination with Django

If you want to build a list page that allows filtering and pagination, you have to get a few separate things to work together. Django provides some tools for pagination, but the documentation doesn't tell us how to make that work with anything else. Similarly, django_filter makes it relatively easy to add filters to a view, but doesn't tell you how to add pagination (or other things) without breaking the filtering.

The heart of the problem is that both features use query parameters, and we need to find a way to let each feature control its own query parameters without breaking the other one.

Filters

Let's start with a review of filtering, with an example of how you might subclass ListView to add filtering. To make it filter the way you want, you need to create a subclass of FilterSet and set filterset_class to that class. (See that link for how to write a filterset.)

class FilteredListView(ListView):
    filterset_class = None

    def get_queryset(self):
        # Get the queryset however you usually would.  For example:
        queryset = super().get_queryset()
        # Then use the query parameters and the queryset to
        # instantiate a filterset and save it as an attribute
        # on the view instance for later.
        self.filterset = self.filterset_class(self.request.GET, queryset=queryset)
        # Return the filtered queryset
        return self.filterset.qs.distinct()

    def get_context_data(self, **kwargs):
        context = super().get_context_data(**kwargs)
        # Pass the filterset to the template - it provides the form.
        context['filterset'] = self.filterset
        return context

Here's an example of how you might create a concrete view to use it:

class BookListView(FilteredListView):
    filterset_class = BookFilterset

And here's part of the template that uses a form created by the filterset to let the user control the filtering.

<h1>Books</h1>
  <form action="" method="get">
    {{ filterset.form.as_p }}
    <input type="submit" />
  </form>

<ul>
    {% for object in object_list %}
        <li>{{ object }}</li>
    {% endfor %}
</ul>

filterset.form is a form that controls the filtering, so we just render that however we want and add a way to submit it.

That's all you need to make a simple filtered view.

Default values for filters

I'm going to digress slightly here, and show a way to give filters default values, so when a user loads a page initially, for example, the items will be sorted with the most recent first. I couldn't find anything about this in the django_filter documentation, and it took me a while to figure out a good solution.

To do this, I override __init__ on my filter set and add default values to the data being passed:

class BookFilterSet(django_filters.FilterSet):
    def __init__(self, data, *args, **kwargs):
        data = data.copy()
        data.setdefault('format', 'paperback')
        data.setdefault('order', '-added')
        super().__init__(data, *args, **kwargs)

I tried some other approaches, but this seemed to work out the simplest, in that it didn't break or complicate things anywhere else.

Combining filtering and pagination

Unfortunately, linking to pages as described above breaks filtering. More specifically, whenever you follow one of those links, the view will forget whatever filtering the user has applied, because that filtering is also controlled by query parameters, and these links don't include the filter's parameters.

So if you're on a page https://example.com/objectlist/?type=paperback and then follow a page link, you'll end up at https://example.com/objectlist/?page=3 when you wanted to be at https://example.com/objectlist/?type=paperback&page=3.

It would be nice if Django helped out with a way to build links that set one query parameter without losing the existing ones, but I found a nice example of a template tag on StackOverflow and modified it slightly into this custom template tag that helps with that:

# <app>/templatetags/my_tags.py
from django import template

register = template.Library()


@register.simple_tag(takes_context=True)
def param_replace(context, **kwargs):
    """
    Return encoded URL parameters that are the same as the current
    request's parameters, only with the specified GET parameters added or changed.

    It also removes any empty parameters to keep things neat,
    so you can remove a parm by setting it to ``""``.

    For example, if you're on the page ``/things/?with_frosting=true&page=5``,
    then

    <a href="/things/?{% param_replace page=3 %}">Page 3</a>

    would expand to

    <a href="/things/?with_frosting=true&page=3">Page 3</a>

    Based on
    https://stackoverflow.com/questions/22734695/next-and-before-links-for-a-django-paginated-query/22735278#22735278
    """
    d = context['request'].GET.copy()
    for k, v in kwargs.items():
        d[k] = v
    for k in [k for k, v in d.items() if not v]:
        del d[k]
    return d.urlencode()

Here's how you can use that template tag to build pagination links that preserve other query parameters used for things like filtering:

{% load my_tags %}

{% if is_paginated %}
  {% if page_obj.has_previous %}
    <a href="?{% param_replace page=1 %}">First</a>
    {% if page_obj.previous_page_number != 1 %}
      <a href="?{% param_replace page=page_obj.previous_page_number %}">Previous</a>
    {% endif %}
  {% endif %}

  Page {{ page_obj.number }} of {{ paginator.num_pages }}

  {% if page_obj.has_next %}
    {% if page_obj.next_page_number != paginator.num_pages %}
      <a href="?{% param_replace page=page_obj.next_page_number %}">Next</a>
    {% endif %}
    <a href="?{% param_replace page=paginator.num_pages %}">Last</a>
  {% endif %}

  <p>Objects {{ page_obj.start_index }}—{{ page_obj.end_index }}</p>
{% endif %}

Now, if you're on a page like https://example.com/objectlist/?type=paperback&page=3, the links will look like ?type=paperback&page=2, ?type=paperback&page=4, etc.

Caktus Group—The Secret Lives of Cakti

Pictured from left: Caktus team members Vinod Kurup, Karen Tracey, and David Ray.

The Caktus team includes expert developers, sharp project managers, and eagle-eyed QA analysts. However, you may not know that there’s more to them than meets the eye. Here’s a peek at how Cakti spend their off-hours.

Vinod Kurup, M.D.

By day Vinod is a mild-mannered developer, but at night he swaps his keyboard for a stethoscope and heads to the hospital. Vinod’s first career was in medicine, and prior to Caktus he worked many years as an MD. While he’s now turned his expertise to programming, he still works part-time as a hospitalist. Now that’s what I call a side hustle.

Karen Tracey, Cat Rescuer

When Karen isn’t busy as both lead developer and technical manager for Caktus, she works extensively with Alley Cats and Angels, a local cat rescue organization dedicated to improving the lives and reducing the population of homeless cats in the Triangle area. She regularly fosters cats and kittens, which is why you sometimes find feline friends hanging out in the Caktus office.

David Ray, Extreme Athlete

Software development and extreme physical endurance training don’t generally go together, but let me introduce you to developer/sales engineer David. When not building solutions for Caktus clients, David straps on a 50-pound pack and completes 24-hour rucking events. Needless to say, he’s one tough Caktus. (Would you believe he’s also a trained opera singer?)

David Ray at a rucking event.

Pictured: David Ray at a recent rucking event.

These are just a few of our illustrious colleagues! Our team also boasts folk musicians, theater artists, sailboat captains, Appalachian cloggers, martial artists, and more.

Want to get to know us better? Drop us a line.

Caktus Group—Django or Drupal for Content Management: Which Fits your Needs?

If you’re building or updating a website, you’re probably wondering about which content management system (CMS) to use. A CMS helps users — particularly non-technical users — to add pages and blog posts, embed videos and images, and incorporate other content into their site.

CMS options

You could go with something quick and do-it-yourself, like WordPress (read more about WordPress) or a drag-and-drop builder like Squarespace. If you need greater functionality, like user account management or asset tracking, or if you’re concerned about security and extensibility, you’ll need a more robust CMS. That means using a framework to build a complex website that can manage large volumes of data and content.

Wait, what’s a framework?

Put simply, a framework is a library of reusable code that is easily edited by a web developer to produce custom products more quickly than coding everything from scratch.

Django and Drupal are both frameworks with dedicated functionality for content management, but there is a key difference between them:

  • Drupal combines aspects of a web application framework with aspects of a CMS
  • Django separates the framework and the CMS

The separation that Django provides makes it easier for content managers to use the CMS because they don’t have to tinker with the technical aspects of the framework. A popular combination is Django and Wagtail, which is our favorite CMS.

I think I’ve heard of Drupal ...

Drupal is open source and built with PHP programming language. For some applications, its customizable templates and quick integrations make it a solid choice. It’s commonly used in higher education settings, among others.

However, Drupal’s predefined templates and plugins can also be its weakness: while they are useful for building a basic site, they are limiting if you want to scale the application. You’ll quickly run into challenges attempting to extend the basic functionality, including adding custom integrations and nonstandard data models.

Other criticisms include:

  • Poor backwards compatibility, particularly for versions earlier than Drupal 7. In this case, updating a Drupal site requires developers to rewrite code for elements of the templates and modules to make them compatible with the newest version. Staying up-to-date is important for security reasons, which can become problematic if the updates are put off too long.
  • Unit testing is difficult due to Drupal’s method of storing configurations in a database, making it difficult to test the effects of changes to sections of the code. Failing to do proper testing may allow errors to make it to the final version of the website.
  • Another database-related challenge lies in how the site configuration is managed. If you’re trying to implement changes on a large website consisting of thousands of individual content items or users, none of the things that usually make this easier — like the ability to view line-by-line site configuration changes during code review — are possible.

What does the above mean for non-technical stakeholders? Development processes are slowed down significantly because developers have to pass massive database files back and forth with low visibility into the changes made by other team members. It also means there is an increased likelihood that errors will reach the public version of your website, creating even more work to fix them.

Caktus prefers Django

Django is used by complex, high-profile websites, including Instagram, Pinterest, and Eventbrite. It’s written in the powerful, open-source Python programming language, which was created specifically to speed up the process of web development. It’s fast, secure, scalable, and intended for use with database-driven web applications.

A huge benefit of Django is more control over customization, plus data can easily be converted. Since it’s built on Python, Django uses a paradigm called object-oriented programming, which makes it easier to manage and manipulate data, troubleshoot errors, and re-use code. It’s also easier for developers to see where changes have been made in the code, simplifying the process of updating the application after it goes live.

How to choose the right tool

Consider the following factors when choosing between Drupal and Django:

  • Need for customization
  • Internal capacity
  • Planning for future updates

Need for customization: If your organization has specific, niche features or functionality that require custom development — for example, data types specific to a library, university, or scientific application — Django is the way to go. It requires more up-front development than template-driven Drupal but allows greater flexibility and customization. Drupal is a good choice if you’re happy to use templates to build your website and don’t need customization.

Internal capacity: Drupal’s steep learning curve means that it may take some time for content managers to get up to speed. In comparison, we’ve run training workshops that get content management teams up and running on Django-based Wagtail in only a day or two. Wagtail’s intuitive user interface makes it easier to manage regular content updates, and the level of customization afforded by Django means the user interface can be developed in a way that feels intuitive to users.

Planning for future updates: Future growth and development should be taken into account when planning a web project. The choices made during the initial project phase will impact the time, expense, and difficulty of future development. As mentioned, Drupal has backwards compatibility challenges, and therefore a web project envisioned as fast-paced and open to frequent updates will benefit from a custom Django solution.

Need a second opinion?

Don’t just take our word for it. Here’s what Brad Busenius at the University of Chicago says about their Django solution:

"[It impacts] the speed and ease at which we can create highly custom interfaces, page types, etc. Instead of trying to bend a general system like Drupal to fit our specific needs, we're able to easily build exactly what we want without any additional overhead. Also, since we're often understaffed, the fact that it's a developer-friendly system helps us a lot. Wagtail has been a very positive experience so far."

The bottom line

Deciding between Django and Drupal comes down to your specific needs and goals, and it’s worth considering the options. That said, based on our 10+ years of experience developing custom websites and web applications, we almost always recommend Django with Wagtail because it’s:

  • Easier to update and maintain
  • More straightforward for content managers to learn and use
  • More efficient with large data sets and complex queries
  • Less likely to let errors slip through the cracks

If you want to consider Django and whether it will suit your next project, we’d be happy to talk it through and share some advice. Get in touch with us.

Caktus Group—Diverse Speaker Line-up for DjangoCon is Sharp

Above: Caktus Account Manager Tim Scales gears up for DjangoCon.

We’re looking forward to taking part in the international gathering of Django enthusiasts at DjangoCon 2018, in San Diego, CA. We’ll be there from October 14 - 19, and we’re proud to attend as sponsors for the ninth year! As such, we’re hosting a mini golf event for attendees (details below).

This year’s speakers are impressive, thanks in part to Erin Mullaney, one of Caktus’ talented developers, who volunteered with DjangoCon’s Program Team. The three-person team, including Developer Jessica Deaton of Wilmington, NC, and Tim Allen, IT Director at The Wharton School, reviewed 257 speaker submissions. They ultimately chose the speakers with the help of a rating system that included community input.

“It was a lot of fun reading the submissions,” said Erin, who will also attend DjangoCon. “I’m really looking forward to seeing the talks this year, especially because I now have a better understanding of how much work goes into the selection process.”

Erin and the program team also created the talk schedule. The roster of speakers includes more women and underrepresented communities due to the DjangoCon diversity initiatives, which Erin is proud to support.

What we’re excited about

Erin said she’s excited about a new State of Django panel that will take place on Wednesday, October 17, which will cap off the conference portion of DjangoCon, before the sprints begin. It should be an informative wrap-up session.

Karen Tracey, our Lead Developer and Technical Manager, is looking forward to hearing “Herding Cats with Django: Technical and social tools to incentivize participation” by Sage Sharp. This talk seems relevant to the continued vibrancy of Django's own development, said Karen, since the core framework and various standard packages are developed with limited funding and rely tremendously on volunteer participation.

Our Account Manager Tim Scales is particularly excited about Tom Dyson’s talk, “Here Come The Robots,” which will explore how people are leveraging Django for machine learning solutions. This is an emerging area of interest for our clients, and one of particular interest to Caktus as we grow our areas of expertise.

Other talks we’re looking forward to include:

Follow us on Twitter @CaktusGroup and #DjangoCon to stay tuned on the talks.

Golf anyone?

If you’re attending DjangoCon, come play a round of mini golf with us. Look for our insert in your conference tote bag. It includes is a free pass to a mini golf outing that we’re hosting at Tiki Town Adventure Golf on Tuesday, October 16, at 7:00 p.m. (please RSVP online). The first round of golf is on us! Whoever shoots the lowest score will win a $100 Amazon gift card.*

No worries if you’re not into mini golf! Instead, find a time to chat with us one-on-one during DjangoCon.

*In the event of a tie, the winner will be selected from a random drawing from the names of those with the lowest score. Caktus employees can play, but are not eligible for prizes.

Caktus Group—Better Python Dependency Management with pip-tools

I recently looked into whether I could use pip-tools to improve my workflow around projects' Python dependencies. My conclusion was that pip-tools would help on some projects, but it wouldn't do everything I wanted, and I couldn't use it everywhere. (I tried pip-tools version 2.0.2 in August 2018. If there are newer versions, they might fix some of the things I ran into when trying pip-tools.)

My problems

What were the problems I wanted to find solutions for, that just pip wasn't handling? Software engineer Kenneth Reitz explains them pretty well in his post, but I'll summarize here.

Let me start by briefly describing the environments I'm concerned with. First is my development environment, where I want to manage the dependencies. Second is the test environment, where I want to know exactly what packages and versions we test with, because then we come to the deployed environment, where I want to use exactly the same Python packages and versions as I've used in development and testing, to be sure no problems are introduced by an unexpected package upgrade.

The way we often handle that is to have a requirements file with every package and its version specified. We might start by installing the packages we know that we need, then saving the output of pip freeze to record all the dependencies that also got installed and their versions. Installing into an empty virtual environment using that requirements file gets us the same packages and versions.

But there are several problems with that approach.

First, we no longer know which packages in that file we originally wanted, and which were pulled in as dependencies. For example, maybe we needed Celery, but installing it pulled in a half-dozen other packages. Later we might decide we don't need Celery anymore and remove it from the requirements file, but we don't know which other packages we can also safely also remove.

Second, it gets very complicated if we want to upgrade some of the packages, for the same reasons.

Third, having to do a complete install of all the packages into an empty virtual environment can be slow, which is especially aggravating when we know little or nothing has changed, but that's the only way to be sure we have exactly what we want.

Requirements

To list my requirements more concisely:

  • Distinguish direct dependencies and versions from incidental
  • Freeze a set of exact packages and versions that we know work
  • Have one command to efficiently update a virtual environment to have exactly the frozen packages at the frozen versions and no other packages
  • Make it reasonably easy to update packages
  • Work with both installing from PyPI, and installing from Git repositories
  • Take advantage of pip's hash checking to give a little more confidence that packages haven't been modified
  • Support multiple sets of dependencies (e.g. dev vs. prod, where prod is not necessarily a subset of dev)
  • Perform reasonably well
  • Be stable

That's a lot of requirements. It turned out that I could meet more of them with pip-tools than just pip, but not all of them, and not for all projects.

Here's what I tried, using pip, virtualenv, and pip-tools.

How to set it up

  1. I put the top-level requirements in requirements.in/*.txt.

    To manage multiple sets of dependencies, we can include "-r file.txt", where "file.txt" is another file in requirements.in, as many times as we want. So we might have a base.txt, a dev.txt that starts with -r base.txt and then adds django-debug-toolbar etc, and a deploy.txt that starts with -r base.txt and then adds gunicorn.

    There's one annoyance that seems minor at this point, but turns out to be a bigger problem: pip-tools only supports URLs in these requirements files if they're marked editable with -e.

# base.txt
Django<2.0
-e git+https://github.com/caktus/django-scribbler@v0.8.0#egg=django-scribbler

# dev.txt
-r base.txt
django-debug-toolbar

# deploy.txt
-r base.txt
gunicorn
  1. Install pip-tools in the relevant virtual environment:
$ <venv>/bin/pip install pip-tools
  1. Compile the requirements as follows:
$ <venv>/bin/pip-compile --output-file requirements/def.txt requirements.in/dev.txt

This looks only at the requirements file(s) we tell it to look at, and not at what's currently installed in the virtual environment. So one unexpected benefit is that pip-compile is faster and simpler than installing everything and then running pip freeze.

The output is a new requirements file at requirements/dev.txt.

pip-compile nicely puts a comment at the top of the output file to tell developers exactly how the file was generated and how to make a newer version of it.

#
# This file is autogenerated by pip-compile
# To update, run:
#
#    pip-compile --output-file requirements/dev.txt requirements.in/dev.txt
#
-e git+https://github.com/caktus/django-scribbler@v0.8.0#egg=django-scribbler
django-debug-toolbar==1.9.1
django==1.11.15
pytz==2018.5
sqlparse==0.2.4           # via django-debug-toolbar
```
  1. Be sure requirements, requirements.in, and their contents are in version control.

How to make the current virtual environment have the same packages and versions

To update your virtual environment to match your requirements file, ensure pip-tools is installed in the desired virtual environment, then:

$ <venv>/bin/pip-sync requirements/dev.txt

And that's all. There's no need to create a new empty virtual environment to make sure only the listed requirements end up installed. If everything is already as we want it, no packages need to be installed at all. Otherwise only the necessary changes are made. And if there's anything installed that's no longer mentioned in our requirements, it gets removed.

Except ...

pip-sync doesn't seem to know how to uninstall the packages that we installed using -e <URL>. I get errors like this:

Can't uninstall 'pkgname1'. No files were found to uninstall.
Can't uninstall 'pkgname2'. No files were found to uninstall.

I don't really know, then, whether pip-sync is keeping those packages up to date. Maybe before running pip-sync, I could just

rm -rf $VIRTUAL_ENV/src

to delete any packages that were installed with -e? But that's ugly and would be easy to forget, so I don't want to do that.

How to update versions

  1. Edit requirements.in/dev.txt if needed.
  2. Run pip-compile again, exactly as before:
$ <venv>/bin/pip-compile--output-file requirements/dev.txt requirements.in/dev.txt
  1. Update the requirements files in version control.

Hash checking

I'd like to use hash checking, but I can't yet. pip-compile can generate hashes for packages we will install from PyPI, but not for ones we install with -e <URL>. Also, pip-sync doesn't check hashes. pip install will check hashes, but if there are any hashes, then it will fail unless all packages have hashes. So if we have any -e <URL> packages, we have to turn off hash generation or we won't be able to pip install with the compiled requirements file. We could still use pip-sync with the requirements file, but since pip-sync doesn't check hashes, there's not much point in having them, even if we don't have any -e packages.

What about pipenv?

Pipenv promises to solve many of these same problems. Unfortunately, it imposes other constraints on my workflow that I don't want. It's also changing too fast at the moment to rely on in production.

Pipenv solves several of the requirements I listed above, but fails on these: It only supports two sets of requirements: base, and base plus dev, not arbitrary sets as I'd like. It can be very slow. It's not (yet?) stable: the interface and behavior is changing constantly, sometimes multiple times in the same day.

It also introduces some new constraints on my workflow. Primarily, it wants to control where the virtual environment is in the filesystem. That both prevents me from putting my virtual environment where I'd like it to be, and prevents me from using different virtual environments with the same working tree.

Shortcomings

pip-tools still has some shortcomings, in addition to the problems with checking hashes I've already mentioned.

Most concerning are the errors from pip-sync when packages have previously been installed using -e <URL>. I feel this is an unresolved issue that needs to be fixed.

Also, I'd prefer not to have to use -e at all when installing from a URL.

This workflow is more complicated than the one we're used to, though no more complicated than we'd have with pipenv, I don't think.

The number and age of open issues in the pip-tools git repository worry me. True, it's orders of magnitude fewer than some projects, but it still suggests to me that pip-tools isn't as well maintained as I might like if I'm going to rely on it in production.

Conclusions

I don't feel that I can trust pip-tools when I need to install packages from Git URLs.

But many projects don't need to install packages from Git URLs, and for those, I think adding pip-tools to my workflow might be a win. I'm going to try it with some real projects and see how that goes for a while.

Josh Johnson—State And Events In CircuitPython: Part 3: State And Microcontrollers And Events (Oh My!)

In this part of the series, we'll apply what we've learned about state to our simple testing code from part one.

Not only will we debounce some buttons without blocking, we'll use state to more efficiently control some LEDs.

We'll also explore what happens when state changes, and how we can take advantage of that to do even more complex things with very little code, using the magic of event detection 🌈 .

All of this will be done in an object-oriented fashion, so we'll learn a lot about OOP as we go along.

Josh Johnson—State And Events In CircuitPython: Part 2: Exploring State And Debouncing The World

In this part of the series, we're going to really dig into what state actually is. We'll use analogies from real life, and then look at how we might model real-life state using Python data structures.

But first, we'll discuss a common problem that all budding electronics engineers have to deal with at some point: "noisy" buttons and how to make them "un-noisy", commonly referred to as "debouncing".

We'll talk about fixing the problem in the worst, but maybe easiest way: by blocking. We'll also talk about why it's bad.

Caktus Group—National Day of Civic Hacking in Durham

Pictured: Simone Sequeira, Senior Product Manager of GetCalFresh, with event attendees at Caktus.

On August 11, I attended the National Day of Civic Hacking hosted by Code for Durham. More than 30 attendees came to the event, hosted in the Caktus Group Tech Space, to collaborate on civic projects that focus on the needs of Durham residents.

National Day of Civic Hacking is a nationwide day of action that brings together civic leaders, local government officials, and community organizers who volunteer their skills to help their local community. Simone Sequeira, Senior Product Manager of GetCalFresh, came from Oakland to participate and present at our Durham event. Simone inspired us with a presentation of GetCalFresh, a project supported by Code for America, that streamlines the application process for food assistance in California. It started as just an idea, and turned into a product used statewide that’s supported by over a half dozen employees. Many Code for Durham projects also start as ideas, and the National Day of Civic Hacking provided an opportunity to turn those ideas into realities.

Laura Biedeger, City of Durham Community Engagement Coordinator, presents at the event at Caktus.

Pictured: Laura Biedeger, a Team Captain at Code for Durham and a co-organizer of the event, speaks to attendees. I'm standing to the left.

Durham Projects

We worked on a variety of projects in Durham, including the following:

One group of designers, programmers, and residents audited the Code for Durham website. The group approached the topic from a user-centered design perspective: they identified and defined user personas and wrote common scenarios of visitors to the site. By the end of the event they had documented the needs of the site and designed mockups for the new site.

Regular volunteers with Code for Durham have been working with the Durham Innovation Team to create an automated texting platform for the Drivers License Restoration Initiative, which aims to support a regular amnesty of driver’s license suspensions in partnership with the Durham District Attorney’s Office. During our event volunteers added a Spanish language track to the platform.

The “Are We Represented?” project focused on voter education: showing how the makeup of County Commissioner boards across the state compare to the population in their county. During the event I worked with Jason Jones, the Analytics and Innovation Manager of Greensboro, to deploy the project to the internet (and we succeeded!).

The Are We Represented group reviews the State Board of Elections data files on a screen.

Pictured: The Are We Represented group reviews State Board of Elections data files.

Another group partnered with End Hunger in Durham, which provides a regularly updated list of food pantries and food producers (gardeners, farmers, grocery stores, bakeries) that regularly donate surplus food. The volunteers reviewed an iOS app they had developed to easily find a pantry, and discussed the development of an Android app.

Join Us Next Time!

The National Day of Civic Hacking gave volunteers a chance to get inspired about new project opportunities, to meet new volunteers, city employees, and to focus on a project for an extended period of time. The projects will continue at Code for Durham’s regularly hosted Meetup at the Caktus Group Tech Space. Volunteers are always welcome, so join us at the next Meetup!

Josh Johnson—State And Events In CircuitPython: Part 1: Setup

This is the first article in a series that explores concepts of state in CircuitPython.

In this installment, we discuss the platform we're using (both CircuitPython and the Adafruit M0/M4 boards that support it), and build a simple circuit for demonstration purposes. We'll also talk a bit about abstraction.

This series is intended for people who are new to Python, programming, and/or microcontrollers, so there's an effort to explain things as thoroughly as possible. However, experience with basic Python would be helpful.

Caktus Group—Complicated Project? Start with our Discovery Workshop Guide

If you ever struggled to implement a complicated development project, starting your next one with a discovery workshop will help. Discovery workshops save you time and money over the course of a project because we help you answer important questions in advance, ensuring that the final product lines up with your primary end goals. Our new guide, Shared Understanding: A Guide to Caktus Discovery Workshops, demonstrates the value of these workshops and why we’ve made them a core component of our client services.

Set Your Project Up for Success

Discovery workshops are vital in defining a project and are an ideal way to overcome the challenges that arise when multiple stakeholders have varying opinions and conflicting visions. By facilitating a discovery workshop, we create a shared understanding of the project and ultimately streamline the development process to ensure that our clients get the best value for their investment. Projects that begin with a discovery phase are more successful for these simple reasons:

  • They cost less because we build the right thing first
  • They’re done faster because we focus on the most valuable features first
  • They have better results because user needs are prioritized from the start

Discovery workshops are part of our best practices for building sharp web apps the right way. We’ve proven that these workshops ensure that projects not only hit their objectives but that they do so on budget, reducing the likelihood of requiring additional work (or money) further down the line.

Get Our Guide

Shared Understanding: A Guide to Caktus Discovery Workshops explains what a Caktus discovery workshop is. It also:

  • Demonstrates how to achieve a shared understanding among stakeholders
  • Provides techniques to uncover discrepancies or areas lacking clarity in the project vision
  • Explains how this knowledge translates into tangible benefits during the project estimation and development process

The guide is an introduction to the aspects of user-centered requirements gathering, which we find most useful at Caktus, and we hope you’ll take a moment to read the free guide:

Caktus Group—ShipIt Day Summer 2018 Recap

On July 27 - 28, we ran our quarterly ShipIt Day here at Caktus. These 24-hour sprints, which we’ve organized since 2012, allow Cakti to explore new projects that expand or increase their skill sets. The event kicked off at 3:00 pm on Thursday and we reconvened at 3:00 pm on Friday to showcase our progress. The goal is to experiment, take risks, and try something new.

Here’s what we got up to this time:

Ship It Day Infographic 2018

Show me more!

Read about previous ShipIt Days in these blog posts.

Caktus Group—Django vs WordPress: How to decide?

In the early stages of a web development project, your first step is a big decision: what’s the right tool for the job? What kind of software makes the most sense for me?

A lot of our clients come to us at this stage, seeking advice on how to approach this decision. And it’s an important one: once you invest in a particular platform, the cost to switch later may be high. You’ll want to make sure you have all the info before making your decision.

Should you use WordPress or Django?

To answer that question, I first need to explain what each of these systems are because we’re actually talking apples and oranges, or maybe apples and fruit salads.

When people say WordPress, they’re referring to the content management system (CMS) that’s used to create and upload the website content. Basically, WordPress is the dashboard through which you organize text and images to display on your website. WordPress is built using the PHP programming language.

Django, on the other hand, is what’s called a web framework. Built on the powerful Python programming language, it’s a set of tools and libraries that can be rapidly deployed to build custom web applications. If your project needs a CMS, Django has a variety of options to choose from. (Our favorite is Wagtail CMS.)

When is WordPress the right fit?

Are you starting a blog, a brochure site, or a small e-commerce site? Go with WordPress.

WordPress is a straightforward and inexpensive tool for a basic site. With a plethora of free themes to download, it’s a quick way to get online and start promoting your content/business/side hustle. Although, if you’re going this route, you may also want to check out options like Squarespace and Wix for even easier site-builders.

If you started with WordPress but realize you want to customize the user experience in ways that the CMS doesn’t facilitate, or you want to connect to custom external applications, then it’s time to look for a more powerful tool.

Why does Caktus prefer Django?

We work with clients to develop custom, scalable, innovative web solutions that deliver sustainable business value, and for that we need the flexibility and power that Django provides.

We work with you to identify specific business challenges that can be addressed through a software solution. WordPress is far too constrained to be useful in this context — we need to customize the application to your specific and unique needs, and for that we need a flexible and comprehensive framework. The complexity of your work requires a powerful solution.

There are also specific things that Django simply does better. Does your project include any of these?

  • Interactive or dynamic content
  • Customizable API integrations
  • Strong data security measures
  • Rapid scalability
  • Custom reports & charts
  • Different views for different types of users
  • Custom validation of data
  • Mobile-friendly applications
  • Fine-grained user permissions

Basically, if you want full control of your user experience, then you’ll definitely want to pick Django.

After all, there’s a reason sites like Instagram, Pinterest, and EventBrite use Django rather than WordPress.

So what’s the next step?

If you think your project falls into the WordPress category, go for it! Head to wordpress.org to learn more.

If you think Django will better suit your needs, we’d be happy to talk it through and share some advice. Get in touch with us now.

Caktus Group—Caktus Embraces GDPR Compliance (and you can, too!)

If you’ve landed on this page, you’re probably here for one of two reasons:

A) You’re a European firm who’s interested in hiring Caktus, in which case: Yes, we’re GDPR compliant! Ask us about it here.

B) You’re trying to figure out what GDPR is and if you need to care about it. If so, read on!

GDPR, or the European General Data Protection Regulation, is a set of rules about how companies collect and use individual personal data. While the law is based in the European Union, it also applies to any companies outside the EU who do business there. That’s why US-based companies like Facebook and Google were paying attention, and why we wanted to ensure compliance as well.

It helped that we were mostly compliant to begin with.

GDPR is designed to ensure that personal data is protected and respected, which falls right in line with our Caktus values. We’re passionate about data security, and we have always maintained strong internal security protections around data that we store or process. Also, we have protections around access to personal data, don’t sell or share contact information, and have always approached our marketing and sales activities from a foundation of respect and honesty.

So what did we change?

To confirm our compliance, we reviewed and updated our internal policies regarding data collection, storage, and usage. Much of this simply involved adding more detail to our current processes. We also examined unusual edge cases and ensured we have policies in place to handle those scenarios. This resulted in a new and more detailed Privacy Policy, plus new internal documentation that explicitly states our guidelines around data collection and management.

We also made a number of small but deliberate tweaks to how we collect and use personal data. For example, instead of pre-checking the “Would you like to subscribe to our newsletter?” checkbox on a contact form, we leave that blank. Therefore, by checking that box, you’re now providing “active consent” to receive our newsletter. You will not be opted-in by default, and that’s the new standard to which we’re holding ourselves. (Speaking of, if you’d like to opt-in to our newsletter, you can do so here.)

We’ve learned a lot, so let’s put that knowledge to use.

Internally, we’ll use our deepened understanding of data protection to ensure that we continue to meet and exceed the law. This will be an ongoing and evolving international conversation, and we intend to stay on the forefront of any changes.

We would also be happy to share what we’ve learned. If you’re exploring a project that will involve the collection or use of personal data, we have an intimate understanding of GDPR compliance to complement our technical expertise. Contact us to start that conversation.

Philip Semanchuk—Thanks to PyOhio 2018!

Thanks to organizers, sponsors, volunteers, and attendees for another great PyOhio!

Here’s the slides from my talk on Python 2 to 3 migration.