Dynamic fields with Flask-WTF
Lately I've been working on a side project that requires dynamically adding or removing fields to a form: ideally, I should be able to add any number of fields and the server should receive the data correctly.
While a combination of FieldList and FormField works great when adding fields in the backend, I wanted to dynamically add and remove fields in the client (through Javascript) without having to send requests to the server until finally submitting the form with all the dynamic fields. Here's how that went.
TL;DR: Example code is available at https://gist.github.com/rmed/def5069419134e9da0713797ccc2cb29.
Edited July 21, 2019: Added persistence to SQLite database and endpoint to display race details.
Edited November 23, 2020: Seeing how several people have asked about this, I have included additional field types to lap subforms (<select>and <textarea>). I also revisited the way the form template is rendered and handled, which translates to easier implementation without having to manually write the HTML. Kudos to all the people in the comments that identified issues and edge cases that I hadn't considered!
Scenario
Let's assume that we have a simple web application used to record the lap times of one or more runners: every time a runner completes a lap, we want to record the amount of time (in minutes) it took them to complete that lap. Once all runners have completed all laps, we will submit the results to the server and store them in a SQLite database.
For this scenario, we could use the following two forms:
MainForm: this would be the form submitted at the end of all laps and contains entries for each of the laps completedLapForm: this sub-form would contain the name of the runner and the time it took them to complete the lap, as well as other miscellaneous fields. These fields will be added and removed dynamically from the frontend
In order to store the data in the database, we will also implement the following models:
Race: will act as container for the lapsLap: stores name and time of each runner, as well as other details such as category and notes on the specific lap
Implementation
This example application will have four files: app.py, templates/macros.html (for separate form rendering macro), templates/index.html (for displaying the form), and templates/show.html (for showing details of a single race).
app.py
The app.py file will contain form definitions, model definitions, and the endpoints used to manage form submission and displaying race details. Starting with the forms:
from flask_wtf import FlaskForm
from wtforms import Form, FieldList, FormField, IntegerField, SelectField, \
StringField, TextAreaField, SubmitField
from wtforms import validators
class LapForm(Form):
"""Subform.
CSRF is disabled for this subform (using `Form` as parent class) because
it is never used by itself.
"""
runner_name = StringField(
'Runner name',
validators=[validators.InputRequired(), validators.Length(max=100)]
)
lap_time = IntegerField(
'Lap time',
validators=[validators.InputRequired(), validators.NumberRange(min=1)]
)
category = SelectField(
'Category',
choices=[('cat1', 'Category 1'), ('cat2', 'Category 2')]
)
notes = TextAreaField(
'Notes',
validators=[validators.Length(max=255)]
)
class MainForm(FlaskForm):
"""Parent form."""
laps = FieldList(
FormField(LapForm),
min_entries=1,
max_entries=20
)
The previous lines implement the forms we mentioned earlier. Note that there will be at least 1 LapForm and a maximum of 20. This means that when accessing the page for the first time, we are guaranteed at least one lap subform and that we can only add a maximum of 20 lap subforms.
Continuing with the database models:
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
class Race(db.Model):
"""Stores races."""
__tablename__ = 'races'
id = db.Column(db.Integer, primary_key=True)
class Lap(db.Model):
"""Stores laps of a race."""
__tablename__ = 'laps'
id = db.Column(db.Integer, primary_key=True)
race_id = db.Column(db.Integer, db.ForeignKey('races.id'))
runner_name = db.Column(db.String(100))
lap_time = db.Column(db.Integer)
category = db.Column(db.String(4))
notes = db.Column(db.String(255))
# Relationship
race = db.relationship(
'Race',
backref=db.backref('laps', lazy='dynamic', collection_class=list)
)
Note that there is a backreference defined in the Lap.race relationship, which would allow us to simply add new laps to an already existing Race object (more on that later).
Now, let's implement the application and the main endpoints:
from flask import Flask, render_template
app = Flask(__name__)
app.config['SECRET_KEY'] = 'sosecret'
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///test.db'
db.init_app(app)
# Create all database tables
db.create_all(app=app)
@app.route('/', methods=['GET', 'POST'])
def index():
form = MainForm()
template_form = LapForm(prefix='laps-_-')
if form.validate_on_submit():
# Create race
new_race = Race()
db.session.add(new_race)
for lap in form.laps.data:
new_lap = Lap(**lap)
# Add to race
new_race.laps.append(new_lap)
db.session.commit()
races = Race.query
return render_template(
'index.html',
form=form,
races=races,
_template=template_form
)
@app.route('/<race_id>', methods=['GET'])
def show_race(race_id):
"""Show the details of a race."""
race = Race.query.filter_by(id=race_id).first()
return render_template(
'show.html',
race=race
)
if __name__ == '__main__':
app.run()
As you can see, when submitting the form in the index endpoint, and after it has been validated, a new_race is created and added to the database session. After that, we create as many laps as received in the request (remember, a maximum of 20!). Given that this data is submitted as a dictionary, it is possible to populate the new lap object using the dictionary as keyword arguments. Finally, since the Race.laps relationship is a list, we can simply append new objects to it.
Note that we are also manually creating a LapForm as follows:
template_form = LapForm(prefix='laps-_-')
And then passing it to index.html as the variable _template. We will talk more about this later on, but this form is going to be used to render the template form used to add new laps directly from the browser.
The prefix argument guarantees that all the fields in template_form are going to be named laps-_-<FIELD>, which matches the naming rules required for dynamic fields to work.
Putting it all together:
# -*- coding: utf-8 -*-
# app.py
from flask import Flask, render_template
from flask_sqlalchemy import SQLAlchemy
from flask_wtf import FlaskForm
from wtforms import Form, FieldList, FormField, IntegerField, SelectField, \
StringField, TextAreaField, SubmitField
from wtforms import validators
class LapForm(Form):
"""Subform.
CSRF is disabled for this subform (using `Form` as parent class) because
it is never used by itself.
"""
runner_name = StringField(
'Runner name',
validators=[validators.InputRequired(), validators.Length(max=100)]
)
lap_time = IntegerField(
'Lap time',
validators=[validators.InputRequired(), validators.NumberRange(min=1)]
)
category = SelectField(
'Category',
choices=[('cat1', 'Category 1'), ('cat2', 'Category 2')]
)
notes = TextAreaField(
'Notes',
validators=[validators.Length(max=255)]
)
class MainForm(FlaskForm):
"""Parent form."""
laps = FieldList(
FormField(LapForm),
min_entries=1,
max_entries=20
)
# Create models
db = SQLAlchemy()
class Race(db.Model):
"""Stores races."""
__tablename__ = 'races'
id = db.Column(db.Integer, primary_key=True)
class Lap(db.Model):
"""Stores laps of a race."""
__tablename__ = 'laps'
id = db.Column(db.Integer, primary_key=True)
race_id = db.Column(db.Integer, db.ForeignKey('races.id'))
runner_name = db.Column(db.String(100))
lap_time = db.Column(db.Integer)
category = db.Column(db.String(4))
notes = db.Column(db.String(255))
# Relationship
race = db.relationship(
'Race',
backref=db.backref('laps', lazy='dynamic', collection_class=list)
)
# Initialize app
app = Flask(__name__)
app.config['SECRET_KEY'] = 'sosecret'
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///test.db'
db.init_app(app)
db.create_all(app=app)
@app.route('/', methods=['GET', 'POST'])
def index():
form = MainForm()
template_form = LapForm(prefix='laps-_-')
if form.validate_on_submit():
# Create race
new_race = Race()
db.session.add(new_race)
for lap in form.laps.data:
new_lap = Lap(**lap)
# Add to race
new_race.laps.append(new_lap)
db.session.commit()
races = Race.query
return render_template(
'index.html',
form=form,
races=races,
_template=template_form
)
@app.route('/<race_id>', methods=['GET'])
def show_race(race_id):
"""Show the details of a race."""
race = Race.query.filter_by(id=race_id).first()
return render_template(
'show.html',
race=race
)
if __name__ == '__main__':
app.run()
templates/macros.html
This file contains the following macro:
{# Render lap form.
This macro is intended to render both regular lap subforms (received from the
server) and the template form used to dynamically add more forms.
Arguments:
- subform: Form object to render
- index: Index of the form. For proper subforms rendered in the form loop,
this should match `loop.index0`, and for the template it should be
'_'
#}
{%- macro render_lap_form(subform, index) %}
<div id="lap-{{ index }}-form" class="{% if index != '_' %}subform{% else %}is-hidden{% endif %}" data-index="{{ index }}">
<div>
{{ subform.runner_name.label }}
{{ subform.runner_name }}
</div>
<div>
{{ subform.lap_time.label }}
{{ subform.lap_time}}
</div>
<div>
{{ subform.category.label }}
{{ subform.category }}
</div>
<div>
{{ subform.notes.label }}
{{ subform.notes }}
</div>
<a class="remove" href="#">Remove</a>
<hr/>
</div>
{%- endmacro %}
This macro receives the following arguments:
subform:LapFormto render in HTML, it can be either a subform included inMainForm.lapsor an independent formindex: The index of the form being rendered
Although the usual way to use this macro is to simply call it while iterating the laps attribute of the main form, it will also be used to render the invisible template form used to add new forms with Javascript. To do so, it is only necessary to specify the value of index as '_'.
The main benefit of rendering the template this way is that we don't have to manually change the HTML each time we change the form definition code in Python, including validators.
templates/index.html
This file has the following main purposes:
- To display current laps in the form (when submitting data)
- To allow the user to add/remove laps as needed
- To serve as index for the races in the database
Let's take a look at the first point:
{% import "macros.html" as macros %}
<html>
<head>
<title>Lap logging</title>
<style>
.is-hidden {
display: none;
}
</style>
</head>
<body>
<a id="add" href="#">Add Lap</a>
<hr/>
{# Show all subforms #}
<form id="lap-form" action="" method="POST" role="form">
{{ form.hidden_tag() }}
<div id="subforms-container">
{% for subform in form.laps %}
{{ macros.render_lap_form(subform, loop.index0) }}
{% endfor %}
</div>
<button type="submit">Send</button>
</form>
{% if form.errors %}
{{ form.errors }}
{% endif %}
{# Form template #}
{{ macros.render_lap_form(_template, '_') }}
{# Show races #}
{% for race in races %}
<p><a href="{{ url_for('show_race', race_id=race.id) }}">Race {{ race.id }}</a></p>
{% endfor %}
</body>
</html>
We end up with something like this:
Note that we are calling the render_lap_form macro introduced before twice. First when rendering the subforms contained within the main form:
<div id="subforms-container">
{% for subform in form.laps %}
{{ macros.render_lap_form(subform, loop.index0) }}
{% endfor %}
</div>
And then when rendering the form template:
{# Form template #}
{{ macros.render_lap_form(_template, '_') }}
If we tried to submit the form we would be sending a single lap subform (the only one present) and we would see the data received by the server.
However, how do we add new forms so that WTForms recognizes the new data? If we take a look at the HTML generated by Flask when rendering the template, we will see that the fields looke like the following:
<div id="lap-0-form" class="subform" data-index="0">
<div>
<label for="laps-0-runner_name">Runner name</label>
<input id="laps-0-runner_name" name="laps-0-runner_name" required="" type="text" value="">
</div>
<div>
<label for="laps-0-lap_time">Lap time</label>
<input id="laps-0-lap_time" name="laps-0-lap_time" required="" type="text" value="">
</div>
<div>
<label for="laps-0-category">Category</label>
<select id="laps-0-category" name="laps-0-category">
<option value="cat1">Category 1</option>
<option value="cat2">Category 2</option>
</select>
</div>
<div>
<label for="laps-0-notes">Notes</label>
<textarea id="laps-0-notes" name="laps-0-notes"></textarea>
</div>
<a class="remove" href="#">Remove</a>
<hr>
</div>
Apparently, WTForms recognizes inputs laps-0-runner_name and laps-0-lap_time as the first LapForm so if we submitted a form which also included laps-1-runner_name and laps-1-lap_time, then WTForms would get both LapForm objects in order.
Remember the macros.render_lap_form(_template, '_') call? That translates to the following HTML:
<div id="lap-_-form" class="is-hidden" data-index="_">
<div>
<label for="laps-_-runner_name">Runner name</label>
<input id="laps-_-runner_name" name="laps-_-runner_name" required="" type="text" value="">
</div>
<div>
<label for="laps-_-lap_time">Lap time</label>
<input id="laps-_-lap_time" name="laps-_-lap_time" required="" type="text" value="">
</div>
<div>
<label for="laps-_-category">Category</label>
<select id="laps-_-category" name="laps-_-category">
<option value="cat1">Category 1</option>
<option value="cat2">Category 2</option
></select>
</div>
<div>
<label for="laps-_-notes">Notes</label>
<textarea id="laps-_-notes" name="laps-_-notes"></textarea>
</div>
<a class="remove" href="#">Remove</a>
<hr>
</div>
Can you spot the difference? Instead of laps-0- appearing in the fields (and labels), it has been replaced with laps-_-, which is the prefix we set before for the template form. This way, we have a unique way of identifying the template and then adjusting the index when inserting a new subform.
Now that we have the template, we need to have the capability of adding and removing fields using the Add and Remove links defined in the template. For this, I will use the following Javascript functions (assuming JQuery is imported, but could easily be adapted for vanilla Javascript):
const ID_RE = /(-)_(-)/;
/**
* Replace the template index of an element (-_-) with the
* given index.
*/
function replaceTemplateIndex(value, index) {
return value.replace(ID_RE, '$1'+index+'$2');
}
/**
* Adjust the indices of form fields when removing items.
*/
function adjustIndices(removedIndex) {
var $forms = $('.subform');
$forms.each(function(i) {
var $form = $(this);
var index = parseInt($form.data('index'));
var newIndex = index - 1;
if (index < removedIndex) {
// Skip
return true;
}
// This will replace the original index with the new one
// only if it is found in the format -num-, preventing
// accidental replacing of fields that may have numbers
// intheir names.
var regex = new RegExp('(-)'+index+'(-)');
var repVal = '$1'+newIndex+'$2';
// Change ID in form itself
$form.attr('id', $form.attr('id').replace(index, newIndex));
$form.data('index', newIndex);
// Change IDs in form fields
$form.find('label, input, select, textarea').each(function(j) {
var $item = $(this);
if ($item.is('label')) {
// Update labels
$item.attr('for', $item.attr('for').replace(regex, repVal));
return;
}
// Update other fields
$item.attr('id', $item.attr('id').replace(regex, repVal));
$item.attr('name', $item.attr('name').replace(regex, repVal));
});
});
}
/**
* Remove a form.
*/
function removeForm() {
var $removedForm = $(this).closest('.subform');
var removedIndex = parseInt($removedForm.data('index'));
$removedForm.remove();
// Update indices
adjustIndices(removedIndex);
}
/**
* Add a new form.
*/
function addForm() {
var $templateForm = $('#lap-_-form');
if ($templateForm.length === 0) {
console.log('[ERROR] Cannot find template');
return;
}
// Get Last index
var $lastForm = $('.subform').last();
var newIndex = 0;
if ($lastForm.length > 0) {
newIndex = parseInt($lastForm.data('index')) + 1;
}
// Maximum of 20 subforms
if (newIndex >= 20) {
console.log('[WARNING] Reached maximum number of elements');
return;
}
// Add elements
var $newForm = $templateForm.clone();
$newForm.attr('id', replaceTemplateIndex($newForm.attr('id'), newIndex));
$newForm.data('index', newIndex);
$newForm.find('label, input, select, textarea').each(function(idx) {
var $item = $(this);
if ($item.is('label')) {
// Update labels
$item.attr('for', replaceTemplateIndex($item.attr('for'), newIndex));
return;
}
// Update other fields
$item.attr('id', replaceTemplateIndex($item.attr('id'), newIndex));
$item.attr('name', replaceTemplateIndex($item.attr('name'), newIndex));
});
// Append
$('#subforms-container').append($newForm);
$newForm.addClass('subform');
$newForm.removeClass('is-hidden');
$newForm.find('.remove').click(removeForm);
}
$(document).ready(function() {
$('#add').click(addForm);
$('.remove').click(removeForm);
});
The functions work as follows:
replaceTemplateIndex: Used to replace-_-in newly added subforms with their appropriate indexadjustIndices: When removing a subform, adjusts the indices of the next subforms so that there is no gap in the numberingremoveForm: Removes a subformaddForm: Copies the template and assigns it the index corresponding to the last subform + 1
The complete templates/index.html file, including a table with all the races in the database, looks as follows:
{# templates/index.html #}
{% import "macros.html" as macros %}
<html>
<head>
<title>Lap logging</title>
{# Import JQuery #}
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
<script>
const ID_RE = /(-)_(-)/;
/**
* Replace the template index of an element (-_-) with the
* given index.
*/
function replaceTemplateIndex(value, index) {
return value.replace(ID_RE, '$1'+index+'$2');
}
/**
* Adjust the indices of form fields when removing items.
*/
function adjustIndices(removedIndex) {
var $forms = $('.subform');
$forms.each(function(i) {
var $form = $(this);
var index = parseInt($form.data('index'));
var newIndex = index - 1;
if (index < removedIndex) {
// Skip
return true;
}
// This will replace the original index with the new one
// only if it is found in the format -num-, preventing
// accidental replacing of fields that may have numbers
// intheir names.
var regex = new RegExp('(-)'+index+'(-)');
var repVal = '$1'+newIndex+'$2';
// Change ID in form itself
$form.attr('id', $form.attr('id').replace(index, newIndex));
$form.data('index', newIndex);
// Change IDs in form fields
$form.find('label, input, select, textarea').each(function(j) {
var $item = $(this);
if ($item.is('label')) {
// Update labels
$item.attr('for', $item.attr('for').replace(regex, repVal));
return;
}
// Update other fields
$item.attr('id', $item.attr('id').replace(regex, repVal));
$item.attr('name', $item.attr('name').replace(regex, repVal));
});
});
}
/**
* Remove a form.
*/
function removeForm() {
var $removedForm = $(this).closest('.subform');
var removedIndex = parseInt($removedForm.data('index'));
$removedForm.remove();
// Update indices
adjustIndices(removedIndex);
}
/**
* Add a new form.
*/
function addForm() {
var $templateForm = $('#lap-_-form');
if ($templateForm.length === 0) {
console.log('[ERROR] Cannot find template');
return;
}
// Get Last index
var $lastForm = $('.subform').last();
var newIndex = 0;
if ($lastForm.length > 0) {
newIndex = parseInt($lastForm.data('index')) + 1;
}
// Maximum of 20 subforms
if (newIndex >= 20) {
console.log('[WARNING] Reached maximum number of elements');
return;
}
// Add elements
var $newForm = $templateForm.clone();
$newForm.attr('id', replaceTemplateIndex($newForm.attr('id'), newIndex));
$newForm.data('index', newIndex);
$newForm.find('label, input, select, textarea').each(function(idx) {
var $item = $(this);
if ($item.is('label')) {
// Update labels
$item.attr('for', replaceTemplateIndex($item.attr('for'), newIndex));
return;
}
// Update other fields
$item.attr('id', replaceTemplateIndex($item.attr('id'), newIndex));
$item.attr('name', replaceTemplateIndex($item.attr('name'), newIndex));
});
// Append
$('#subforms-container').append($newForm);
$newForm.addClass('subform');
$newForm.removeClass('is-hidden');
$newForm.find('.remove').click(removeForm);
}
$(document).ready(function() {
$('#add').click(addForm);
$('.remove').click(removeForm);
});
</script>
<style>
.is-hidden {
display: none;
}
</style>
</head>
<body>
<a id="add" href="#">Add Lap</a>
<hr/>
{# Show all subforms #}
<form id="lap-form" action="" method="POST" role="form">
{{ form.hidden_tag() }}
<div id="subforms-container">
{% for subform in form.laps %}
{{ macros.render_lap_form(subform, loop.index0) }}
{% endfor %}
</div>
<button type="submit">Send</button>
</form>
{% if form.errors %}
{{ form.errors }}
{% endif %}
{# Form template #}
{{ macros.render_lap_form(_template, '_') }}
{# Show races #}
{% for race in races %}
<p><a href="{{ url_for('show_race', race_id=race.id) }}">Race {{ race.id }}</a></p>
{% endfor %}
</body>
</html>
templates/show.html
For this template we simply want to display the results of a specific race (through the /<race_id> URL). Therefore, a simple table should fit this purpose perfectly:
{# templates/show.html #}
<html>
<head>
<title>Race details</title>
</head>
<body>
<a href="{{ url_for('index') }}">Back to index</a>
{% if not race %}
<p>Could not find race details</p>
{% else %}
<table>
<thead>
<tr>
<th>Runner name</th>
<th>Lap time</th>
<th>Category</th>
<th>Notes</th>
</tr>
</thead>
<tbody>
{% for lap in race.laps %}
<tr>
<td>{{ lap.runner_name }}</td>
<td>{{ lap.lap_time }}</td>
<td>
{%- if lap.category == 'cat1' %}
Category 1
{%- elif lap.category == 'cat2' %}
Category 2
{%- else %}
Unknown
{%- endif %}
</td>
<td>{{ lap.notes }}</td>
</tr>
{% endfor%}
</tbody>
</table>
{% endif %}
</body>
</html>
Testing
In order to test the application, simply run python3 app.py and try adding and removing fields. After submitting the form, check the Race link added at the bottom of the page, which should look like this:
