Implement type-specific forms for records

[alehaa]

Is your feature request related to a problem? Please describe.

Currently, I use a custom plugin for managing DNS and IP blocklists. To reduce maintenance overhead and leverage existing APIs, I’d like to switch to DNS Response Policy Zones, which also cover these types of blocklists. However, this requires users to understand the semantics of policy records to implement the desired behavior of the blocklist.

Similarly, to avoid errors, users need to understand the structure of more complex DNS records, such as SRV and NAPTR.

Describe the solution you'd like

I propose providing custom forms for more complex DNS record types to help users input the correct data. Once saved, the appropriate DNS value will be generated from the form fields and stored in the database. When editing existing DNS record objects, the record value will be automatically broken down before being displayed as individual form fields.

This modification does not require any changes to the database or API. My primary intention is to implement visual changes to simplify the process.

In the first iteration I propose implementing this change for these record types:

• CAA
• MX
• NAPTR
• SRV
• TLSA

I believe there’s more potential in this feature, which makes this feature request somewhat complex. For instance, there are various types of TXT records that require a specific structure (e.g., SPF, DMARC, or RPZ, as mentioned earlier). An abstract form model should simplify the registration of new formats, thereby reducing maintenance overhead and improving data quality. This logic should be implemented in a subsequent feature request.

Describe alternatives you've considered

One could argue whether DNS Response Policy Zones is a feature of this plugin and should be implemented elsewhere. However, since this technique heavily relies on DNS for its service, I believe it falls under the category of DNS record values like MX or SRV.

With this feature request, I intend to enhance user experience not only for implementing RPZ but also for managing various complex DNS records.

Additional context

In a proof of concept, I’ve already verified that the RecordEditView form can be dynamically selected by replacing the form attribute with a static method:

@register_model_view(Record, "add", detail=False)
@register_model_view(Record, "edit")
class RecordEditView(generic.ObjectEditView):
    queryset = Record.objects.filter(managed=False).prefetch_related(
        "zone", "ptr_record"
    )
    
    @staticmethod
    def form(instance: Record | None, *args, **kwargs):
        return RecordForm(instance=instance, *args, **kwargs)

The appropriate form should be selected based on the instance’s record type. For creating new records, a query parameter can be used instead. This parameter will be passed by a drop-down menu on the Add Record button in the zone record view. If no specific form is available, a fallback to the generic form will always be used.

[peteeckel]

Hi @alehaa, thanks for this feature suggestion.

I've considered that kind of input form improvement a couple of times. There are some issues that make changing the forms in the way you suggested a bit more complicated, and IMHO a bit more complicated for me to put effort into it at the moment - there are more pressing enhancements, e.g. catalog zones, and it's merely usability feature, albeit a very sensible one.

One of the problems is that the input form for records needs to be dynamic to support this kind of feature, i.e. the other form fields (e.g. priority/target for an MX record) need to be displayed only if the "type" field is "MX".

This has been solved by the NetBox team for DynamicModelChoiceField and DynamicModelMultipleChoiceField, which can use the HTMXWidget to do a refresh of the model form if their content changes. However, that widget does not work for ChoiceField (I tried :-)).

Changing this probably requires some tweaking of the JS code, and that's something I shirk away from - I hate JavaScript from the bottom of my heart.

That said - I probably will not do it in the near future, but if you want to have a got at it you're very welcome to do so!

Your sample code should work well, but only if the type of the record is already known. It does not cover the case of creating a new record (where type is not defined yet) or where you want to change the type of a record (which is admittedly a rare case and can be ruled out by making type immutable for existing records). Dynamic forms are key to this functionality.

[alehaa]

I delved into the NetBox core code and discovered that adding HTMXSelect to a form field triggers a reload whenever there are changes. This means that no changes are required in JavaScript, and I would like to avoid making such code changes.

 class RecordForm(TenancyForm, NetBoxModelForm):
     type = forms.ChoiceField(
         choices=add_blank_choice(RecordSelectableTypeChoices),
         required=True,
         label=_("Type"),
+        widget=HTMXSelect(),
     )

The selected type will be passed to the form field when it’s created, enabling the method described above to modify the form class that’s used.

@register_model_view(Record, "add", detail=False)
@register_model_view(Record, "edit")
class RecordEditView(generic.ObjectEditView):
    queryset = Record.objects.filter(managed=False).prefetch_related(
        "zone", "ptr_record"
    )

    @staticmethod
    def form(instance: Record | None, initial: dict, *args, **kwargs):
        # Load specialized form if available.
        record_type = initial.get("type")
        if record_type == "MX":
            return RecordFormMX(instance=instance, initial=initial, *args, **kwargs)

        # Fallback to form with plain value field.
        return RecordForm(instance=instance, initial=initial, *args, **kwargs)

[peteeckel]

Hi @alehaa, correct, that does work for DynamicModelChoiceField and DynamicModelMultipleChoiceField.

It does not, however, for ChoiceField, which would be required for the type field - I tried that a while back, and found no viable solution at that time until recently.

There is, however, a way to handle this, albeit not a very elegant one:

DYNAMIC_ATTRIBUTES = {
    "hx-get": ".",
    "hx-include": "#form_fields",
    "hx-target": "#form_fields",
}

class RecordForm(
    fields = (
        [...]
        "value",
        "priority",
        "target",
    )

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

        self.fields["type"].widget.attrs.update(DYNAMIC_ATTRIBUTES)

        if (type := get_field_value(self, "type")) == RecordTypeChoices.MX:
            del self.fields["value"]
        else:
            del self.fields["priority"]
            del self.fields["target"]

    [...]

The idea is to define all fields that are required by specific RRTYPEs initially, and delete all of them if the value of the type field does not match a selection that requires them. The clean() function is then used to reassemble the value field from the specific fields.

The part about it that I don't particularly like is the embedding of HTMX attributes in the form code. Without it (using the HTMXSelect widget), the form field does not get a value when the form is evaluated, at least that was the case when I tried last. It seems to be the only way to get it done, however, as the core project goes down exactly the same road (see, for example, netbox/dcim/forms/model_forms.py).

This has the advantage of not needing specific forms for each and every RRTYPE and is what I clearly prefer to the solution with conditional forms.

Caveat: There are several dozens of records having structured values, and if there is a special form for one of them I would prefer to have it for all ... which means a whole lot of code, and as many specific tests as there are such RRTYPEs. Some even have multiple options for the sub-fields, further complicating the issue ... so if someone wants to invest the time, please do so (and keep in touch so we don't waste effort), but I have other things I'm planning to implement (e.g. catalog zone suppotz) before considering that.

[peteeckel]

By the way, "type-agnostic" would be "not knowing about the type", i.e. what we have now ... I changed the title to better reflect the idea.

[alehaa]

It does not, however, for ChoiceField, which would be required for the type field - I tried that a while back, and found no viable solution at that time until recently.

@peteeckel, are you certain about that? I attempted this a few minutes ago in a local deployment using NetBox version 4.4.7. A proof of concept is available here: alehaa@b999fe1 This is not the final implementation, but the logic should generally work.

There are several dozens of records having structured values, and if there is a special form for one of them I would prefer to have it for all …

I understand the issue. Would this feature be a „raise a PR if you want another form“ type of solution, or something like „we need a generic solution“? If it’s accepted, I considered implementing base classes for both forms and tests to simplify the process of adding new forms. However, the general maintenance problem persists.

[peteeckel]

Hi @alehaa,

regarding your proof of concept - have you tried switching the record types back and forth and seeing if the "assisted" fields change each time? My experience is that it works if the type is pre-populated, but once you change it it can no longer be retrieved in the __init__() method. It may help that you use a different class for MX record forms, but that would IMHO be an even uglier solution that mine (and mine is already pretty ugly and IMHO not production-ready for this case).

I would actually prefer a generic solution over a "one PR per record type" one. My main issue with the latter is that it creates a ton of slightly different variations in the code, and probably many duplications as well. My favourite idea is to create some kind of register that maps RRTYPEs to sets of fields with data types and optional validation methods (many are already provided by dnspython, but not all - domain names can be host names or not, for example, depending on the RRTYPE), so that a generic RecordForm can be used that switches between the specific field sets when the type is changed.

By the way, digesting your changes would be way easier if you hadn't changed the order of import lines, tuple elements, etc., which creates a whole lot of unnecessary diff lines.