Merge pull request #265 from netbox-community/develop

Release 0.23.0

.dockerignore

@@ -7,5 +7,4 @@ build*
 docker-compose.override.yml
 .netbox/.git*
 .netbox/.travis.yml
-.netbox/docs
 .netbox/scripts

README.md

@@ -12,7 +12,8 @@
 [The Github repository](netbox-docker-github) houses the components needed to build Netbox as a Docker container.
 Images are built using this code and are released to [Docker Hub][netbox-dockerhub] once a day.
 
-Do you have any questions? Before opening an issue on Github, please join the [Network To Code][ntc-slack] Slack and ask for help in our [`#netbox-docker`][netbox-docker-slack] channel.
+Do you have any questions?
+Before opening an issue on Github, please join the [Network To Code][ntc-slack] Slack and ask for help in our [`#netbox-docker`][netbox-docker-slack] channel.
 
 [github-stargazers]: https://github.com/netbox-community/netbox-docker/stargazers
 [github-release]: https://github.com/netbox-community/netbox-docker/releases
@@ -25,16 +26,18 @@ Do you have any questions? Before opening an issue on Github, please join the [N
 
 ## Docker Tags
 
-* `vX.Y.Z`: Release builds, built from [releases of Netbox][netbox-releases].
-* `latest`: Release builds, built from [`master` branch of Netbox][netbox-master].
-* `snapshot`: Pre-release builds, built from the [`develop` branch of Netbox][netbox-develop].
-* `develop-X.Y`: Pre-release builds, built from the corresponding [branch of Netbox][netbox-branches].
+* `vX.Y.Z`: These are release builds, automatically built from [the corresponding releases of Netbox][netbox-releases].
+* `latest`: These are release builds, automatically built from [the `master` branch of Netbox][netbox-master].
+* `snapshot`: These are pre-release builds, automatically built from the [`develop` branch of Netbox][netbox-develop].
+* `develop-X.Y`: These are pre-release builds, automatically built from the corresponding [branch of Netbox][netbox-branches].
 
-Then there is currently one extra tags for each of the above labels:
+Then there is currently one extra tags for each of the above tags:
 
 * `-ldap`: Contains additional dependencies and configurations for connecting Netbox to an LDAP directroy.
   [Learn more about that in our wiki][netbox-docker-ldap].
 
+New images are built and published automatically every ~24h.
+
 [netbox-releases]: https://github.com/netbox-community/netbox/releases
 [netbox-master]: https://github.com/netbox-community/netbox/tree/master
 [netbox-develop]: https://github.com/netbox-community/netbox/tree/develop
@@ -43,42 +46,50 @@ Then there is currently one extra tags for each of the above labels:
 
 ## Quickstart
 
-To get Netbox up and running in Docker:
+To get Netbox Docker up and running run the following commands.
+There is a more complete [_Getting Started_ guide on our wiki][wiki-getting-started] which explains every step.
 
 ```bash
 git clone -b release https://github.com/netbox-community/netbox-docker.git
 cd netbox-docker
+tee netbox-docker.override.yml <<EOF
+version: '3.4'
+services:
+  nginx:
+    ports:
+      - 8000:8080
+EOF
 docker-compose pull
-docker-compose up -d
+docker-compose up
 ```
 
-The application will be available after a few minutes.
-Use `docker-compose port nginx 8080` to find out where to connect to.
-
-```bash
-$ echo "http://$(docker-compose port nginx 8080)/"
-http://0.0.0.0:32768/
-
-# Open netbox in your default browser on macOS:
-$ open "http://$(docker-compose port nginx 8080)/"
-
-# Open netbox in your default browser on (most) linuxes:
-$ xdg-open "http://$(docker-compose port nginx 8080)/" &>/dev/null &
-```
-
-Alternatively, use something like [Reception][docker-reception] to connect to _docker-compose_ projects.
-
+The whole application will be available after a few minutes.
+Open the URL `http://0.0.0.0:8000/` in a web-browser.
+You should see the Netbox homepage.
+In the top-right corner you can login.
 The default credentials are:
 
 * Username: **admin**
 * Password: **admin**
 * API Token: **0123456789abcdef0123456789abcdef01234567**
 
-There is a more complete [Getting Started guide on our Wiki][wiki-getting-started].
-
 [wiki-getting-started]: https://github.com/netbox-community/netbox-docker/wiki/Getting-Started
 [docker-reception]: https://github.com/nxt-engineering/reception
 
+## Documentation
+
+Please refer [to our wiki on Github][netbox-docker-wiki] for further information on how to use this Netbox Docker image properly.
+It covers advanced topics such as using secret files, deployment to Kubernetes as well as NAPALM and LDAP configuration.
+
+[netbox-docker-wiki]: https://github.com/netbox-community/netbox-docker/wiki/
+
+## Getting Help
+
+Please join [our Slack channel `#netbox-docker`][netbox-docker-slack] on the [Network To Code Slack][ntc-slack].
+It's free to use and there are almost always people online that can help.
+
+If you need help with using Netbox or developing for it or against it's API you may find the `#netbox` channel on the same Slack instance very helpful.
+
 ## Dependencies
 
 This project relies only on *Docker* and *docker-compose* meeting these requirements:
@@ -88,17 +99,10 @@ This project relies only on *Docker* and *docker-compose* meeting these requirem
 
 To check the version installed on your system run `docker --version` and `docker-compose --version`.
 
-## Documentation
+## Use a Specific Netbox Version
 
-Please refer [to our wiki on Github][netbox-docker-wiki] for further information on how to use this Netbox Docker image properly.
-It covers advanced topics such as using secret files, deployment to Kubernetes as well as NAPALM and LDAP configuration.
-
-[netbox-docker-wiki]: https://github.com/netbox-community/netbox-docker/wiki/
-
-## Netbox Version
-
-The `docker-compose.yml` file is prepared to run a specific version of Netbox.
-To use this feature, set the environment-variable `VERSION` before launching `docker-compose`, as shown below.
+The `docker-compose.yml` file is prepared to run a specific version of Netbox, instead of `latest`.
+To use this feature, set and export the environment-variable `VERSION` before launching `docker-compose`, as shown below.
 `VERSION` may be set to the name of
 [any tag of the `netboxcommunity/netbox` Docker image on Docker Hub][netbox-dockerhub].
 
@@ -141,10 +145,6 @@ For more details on custom builds [consult our wiki][netbox-docker-wiki-build].
 
 [netbox-docker-wiki-build]: https://github.com/netbox-community/netbox-docker/wiki/Build
 
-### Pre-made Docker Images
-
-New Docker images are built and published every 24h.
-
 ## Tests
 
 We have a test script.

VERSION

@@ -1 +1 @@
-0.22.0
+0.23.0

configuration/configuration.py

@@ -170,6 +170,15 @@ PAGINATE_COUNT = int(os.environ.get('PAGINATE_COUNT', 50))
 # When determining the primary IP address for a device, IPv6 is preferred over IPv4 by default. Set this to True to
 # prefer IPv4 instead.
 PREFER_IPV4 = os.environ.get('PREFER_IPV4', 'False').lower() == 'true'
+
+# This determines how often the GitHub API is called to check the latest release of NetBox in seconds. Must be at least 1 hour.
+RELEASE_CHECK_TIMEOUT = os.environ.get('RELEASE_CHECK_TIMEOUT', 24 * 3600)
+
+# This repository is used to check whether there is a new release of NetBox available. Set to None to disable the
+# version check or use the URL below to check for release in the official NetBox repository.
+# https://api.github.com/repos/netbox-community/netbox/releases
+RELEASE_CHECK_URL = os.environ.get('RELEASE_CHECK_URL', None)
+
 # The file path where custom reports will be stored. A trailing slash is not needed. Note that the default value of
 # this setting is derived from the installed location.
 REPORTS_ROOT = os.environ.get('REPORTS_ROOT', '/etc/netbox/reports')

docker-compose.test.yml

@@ -5,6 +5,7 @@ services:
     depends_on:
     - postgres
     - redis
+    - redis-cache
     env_file: env/netbox.env
     user: '101'
     volumes:
@@ -34,8 +35,15 @@ services:
     command:
     - sh
     - -c # this is to evaluate the $REDIS_PASSWORD from the env
-    - redis-server --requirepass $$REDIS_PASSWORD ## $$ because of docker-compose
+    - redis-server --appendonly yes --requirepass $$REDIS_PASSWORD ## $$ because of docker-compose
     env_file: env/redis.env
+  redis-cache:
+    image: redis:5-alpine
+    command:
+    - sh
+    - -c # this is to evaluate the $REDIS_PASSWORD from the env
+    - redis-server --requirepass $$REDIS_PASSWORD ## $$ because of docker-compose
+    env_file: env/redis-cache.env
 volumes:
   netbox-static-files:
     driver: local

docker-compose.yml

@@ -57,7 +57,7 @@ services:
     - sh
     - -c # this is to evaluate the $REDIS_PASSWORD from the env
     - redis-server --requirepass $$REDIS_PASSWORD ## $$ because of docker-compose
-    env_file: env/redis.env
+    env_file: env/redis-cache.env
 volumes:
   netbox-static-files:
     driver: local

env/netbox.env

@@ -20,7 +20,7 @@ REDIS_DATABASE=0
 REDIS_SSL=false
 REDIS_CACHE_HOST=redis-cache
 REDIS_CACHE_PASSWORD=t4Ph722qJ5QHeQ1qfu36
-REDIS_CACHE_DATABASE=0
+REDIS_CACHE_DATABASE=1
 REDIS_CACHE_SSL=false
 SECRET_KEY=r8OwDznj!!dci#P9ghmRfdu1Ysxm0AiPeDCQhKE+N_rClfWNj
 SKIP_STARTUP_SCRIPTS=false
@@ -30,3 +30,4 @@ SUPERUSER_EMAIL=admin@example.com
 SUPERUSER_PASSWORD=admin
 SUPERUSER_API_TOKEN=0123456789abcdef0123456789abcdef01234567
 WEBHOOKS_ENABLED=true
+RELEASE_CHECK_URL=https://api.github.com/repos/netbox-community/netbox/releases

startup_scripts/000_users.py

@@ -1,19 +1,13 @@
-from django.contrib.auth.models import Permission, Group, User
-from users.models import Token
-
-from ruamel.yaml import YAML
-from pathlib import Path
 import sys
 
-file = Path('/opt/netbox/initializers/users.yml')
-if not file.is_file():
+from django.contrib.auth.models import Group, User
+from startup_script_utils import load_yaml, set_permissions
+from users.models import Token
+
+users = load_yaml('/opt/netbox/initializers/users.yml')
+if users is None:
   sys.exit()
 
-with file.open('r') as stream:
-  yaml=YAML(typ='safe')
-  users = yaml.load(stream)
-
-  if users is not None:
 for username, user_details in users.items():
   if not User.objects.filter(username=username):
     user = User.objects.create_user(
@@ -26,17 +20,4 @@ with file.open('r') as stream:
       Token.objects.create(user=user, key=user_details['api_token'])
 
     yaml_permissions = user_details.get('permissions', [])
-        if yaml_permissions:
-          subject = user.user_permissions
-          subject.clear()
-          for yaml_permission in yaml_permissions:
-            if '*' in yaml_permission:
-              permission_filter = '^' + yaml_permission.replace('*','.*') + '$'
-              permissions = Permission.objects.filter(codename__iregex=permission_filter)
-              print("  ⚿ Granting", permissions.count(), "permissions matching '" + yaml_permission + "'")
-            else:
-              permissions = Permission.objects.filter(codename=yaml_permission)
-              print("  ⚿ Granting permission", yaml_permission)
-            
-            for permission in permissions:
-              subject.add(permission)
+    set_permissions(user.user_permissions, yaml_permissions)

startup_scripts/010_groups.py

@@ -1,17 +1,12 @@
-from django.contrib.auth.models import Permission, Group, User
-from ruamel.yaml import YAML
-from pathlib import Path
 import sys
 
-file = Path('/opt/netbox/initializers/groups.yml')
-if not file.is_file():
+from django.contrib.auth.models import Group, User
+from startup_script_utils import load_yaml, set_permissions
+
+groups = load_yaml('/opt/netbox/initializers/groups.yml')
+if groups is None:
   sys.exit()
 
-with file.open('r') as stream:
-  yaml=YAML(typ='safe')
-  groups = yaml.load(stream)
-
-  if groups is not None:
 for groupname, group_details in groups.items():
   group, created = Group.objects.get_or_create(name=groupname)
 
@@ -25,17 +20,4 @@ with file.open('r') as stream:
       user.groups.add(group)
 
   yaml_permissions = group_details.get('permissions', [])
-      if yaml_permissions:
-        subject = group.permissions
-        subject.clear()
-        for yaml_permission in yaml_permissions:
-          if '*' in yaml_permission:
-            permission_filter = '^' + yaml_permission.replace('*','.*') + '$'
-            permissions = Permission.objects.filter(codename__iregex=permission_filter)
-            print("  ⚿ Granting", permissions.count(), "permissions matching '" + yaml_permission + "'")
-          else:
-            permissions = Permission.objects.filter(codename=yaml_permission)
-            print("  ⚿ Granting permission", yaml_permission)
-
-          for permission in permissions:
-            subject.add(permission)
+  set_permissions(group.permissions, yaml_permissions)

startup_scripts/020_custom_fields.py

@@ -1,7 +1,6 @@
 from extras.models import CustomField, CustomFieldChoice
 
-from ruamel.yaml import YAML
-from pathlib import Path
+from startup_script_utils import load_yaml
 import sys
 
 def get_class_for_class_path(class_path):
@@ -13,15 +12,11 @@ def get_class_for_class_path(class_path):
   clazz = getattr(module, class_name)
   return ContentType.objects.get_for_model(clazz)
 
-file = Path('/opt/netbox/initializers/custom_fields.yml')
-if not file.is_file():
+customfields = load_yaml('/opt/netbox/initializers/custom_fields.yml')
+
+if customfields is None:
   sys.exit()
 
-with file.open('r') as stream:
-  yaml = YAML(typ='safe')
-  customfields = yaml.load(stream)
-
-  if customfields is not None:
 for cf_name, cf_details in customfields.items():
   custom_field, created = CustomField.objects.get_or_create(name = cf_name)
 

startup_scripts/030_regions.py

@@ -1,21 +1,16 @@
 from dcim.models import Region
-from ruamel.yaml import YAML
-from pathlib import Path
+from startup_script_utils import load_yaml
 import sys
 
-file = Path('/opt/netbox/initializers/regions.yml')
-if not file.is_file():
-  sys.exit()
+regions = load_yaml('/opt/netbox/initializers/regions.yml')
 
-with file.open('r') as stream:
-  yaml=YAML(typ='safe')
-  regions = yaml.load(stream)
+if regions is None:
+  sys.exit()
 
 optional_assocs = {
   'parent': (Region, 'name')
 }
 
-  if regions is not None:
 for params in regions:
 
   for assoc, details in optional_assocs.items():

startup_scripts/040_sites.py

@@ -1,24 +1,19 @@
 from dcim.models import Region, Site
 from extras.models import CustomField, CustomFieldValue
 from tenancy.models import Tenant
-from ruamel.yaml import YAML
-from pathlib import Path
+from startup_script_utils import load_yaml
 import sys
 
-file = Path('/opt/netbox/initializers/sites.yml')
-if not file.is_file():
-  sys.exit()
+sites = load_yaml('/opt/netbox/initializers/sites.yml')
 
-with file.open('r') as stream:
-  yaml = YAML(typ='safe')
-  sites = yaml.load(stream)
+if sites is None:
+  sys.exit()
 
 optional_assocs = {
   'region': (Region, 'name'),
   'tenant': (Tenant, 'name')
 }
 
-  if sites is not None:
 for params in sites:
   custom_fields = params.pop('custom_fields', None)
 

startup_scripts/050_manufacturers.py

@@ -1,17 +1,12 @@
 from dcim.models import Manufacturer
-from ruamel.yaml import YAML
-from pathlib import Path
+from startup_script_utils import load_yaml
 import sys
 
-file = Path('/opt/netbox/initializers/manufacturers.yml')
-if not file.is_file():
+manufacturers = load_yaml('/opt/netbox/initializers/manufacturers.yml')
+
+if manufacturers is None:
   sys.exit()
 
-with file.open('r') as stream:
-  yaml = YAML(typ='safe')
-  manufacturers = yaml.load(stream)
-
-  if manufacturers is not None:
 for params in manufacturers:
   manufacturer, created = Manufacturer.objects.get_or_create(**params)
 

startup_scripts/060_device_types.py

@@ -1,17 +1,13 @@
 from dcim.models import DeviceType, Manufacturer, Region
 from tenancy.models import Tenant
 from extras.models import CustomField, CustomFieldValue
-from ruamel.yaml import YAML
-from pathlib import Path
+from startup_script_utils import load_yaml
 import sys
 
-file = Path('/opt/netbox/initializers/device_types.yml')
-if not file.is_file():
-  sys.exit()
+device_types = load_yaml('/opt/netbox/initializers/device_types.yml')
 
-with file.open('r') as stream:
-  yaml = YAML(typ='safe')
-  device_types = yaml.load(stream)
+if device_types is None:
+  sys.exit()
 
 required_assocs = {
   'manufacturer': (Manufacturer, 'name')
@@ -22,7 +18,6 @@ with file.open('r') as stream:
   'tenant': (Tenant, 'name')
 }
 
-  if device_types is not None:
 for params in device_types:
   custom_fields = params.pop('custom_fields', None)
 

startup_scripts/070_rack_roles.py

@@ -1,19 +1,14 @@
 from dcim.models import RackRole
-from ruamel.yaml import YAML
 from utilities.forms import COLOR_CHOICES
 
-from pathlib import Path
+from startup_script_utils import load_yaml
 import sys
 
-file = Path('/opt/netbox/initializers/rack_roles.yml')
-if not file.is_file():
+rack_roles = load_yaml('/opt/netbox/initializers/rack_roles.yml')
+
+if rack_roles is None:
   sys.exit()
 
-with file.open('r') as stream:
-  yaml=YAML(typ='safe')
-  rack_roles = yaml.load(stream)
-
-  if rack_roles is not None:
 for params in rack_roles:
   if 'color' in params:
     color = params.pop('color')

startup_scripts/075_rack_groups.py

@@ -1,22 +1,16 @@
 from dcim.models import Site,RackGroup
-from ruamel.yaml import YAML
-
-from pathlib import Path
+from startup_script_utils import load_yaml
 import sys
 
-file = Path('/opt/netbox/initializers/rack_groups.yml')
-if not file.is_file():
-  sys.exit()
+rack_groups = load_yaml('/opt/netbox/initializers/rack_groups.yml')
 
-with file.open('r') as stream:
-  yaml=YAML(typ='safe')
-  rack_groups= yaml.load(stream)
+if rack_groups is None:
+  sys.exit()
 
 required_assocs = {
   'site': (Site, 'name')
 }
 
-  if rack_groups is not None:
 for params in rack_groups:
 
   for assoc, details in required_assocs.items():

startup_scripts/080_racks.py

@@ -1,17 +1,13 @@
 from dcim.models import Site, RackRole, Rack, RackGroup
 from tenancy.models import Tenant
 from extras.models import CustomField, CustomFieldValue
-from ruamel.yaml import YAML
-from pathlib import Path
+from startup_script_utils import load_yaml
 import sys
 
-file = Path('/opt/netbox/initializers/racks.yml')
-if not file.is_file():
-  sys.exit()
+racks = load_yaml('/opt/netbox/initializers/racks.yml')
 
-with file.open('r') as stream:
-  yaml = YAML(typ='safe')
-  racks = yaml.load(stream)
+if racks is None:
+  sys.exit()
 
 required_assocs = {
   'site': (Site, 'name')
@@ -23,7 +19,6 @@ with file.open('r') as stream:
   'group': (RackGroup, 'name')
 }
 
-  if racks is not None:
 for params in racks:
   custom_fields = params.pop('custom_fields', None)
 

startup_scripts/090_device_roles.py

@@ -1,19 +1,13 @@
 from dcim.models import DeviceRole
-from ruamel.yaml import YAML
 from utilities.forms import COLOR_CHOICES
-
-from pathlib import Path
+from startup_script_utils import load_yaml
 import sys
 
-file = Path('/opt/netbox/initializers/device_roles.yml')
-if not file.is_file():
+device_roles = load_yaml('/opt/netbox/initializers/device_roles.yml')
+
+if device_roles is None:
   sys.exit()
 
-with file.open('r') as stream:
-  yaml=YAML(typ='safe')
-  device_roles = yaml.load(stream)
-
-  if device_roles is not None:
 for params in device_roles:
 
   if 'color' in params:

startup_scripts/100_platforms.py

@@ -1,22 +1,16 @@
 from dcim.models import Manufacturer, Platform
-from ruamel.yaml import YAML
-
-from pathlib import Path
+from startup_script_utils import load_yaml
 import sys
 
-file = Path('/opt/netbox/initializers/platforms.yml')
-if not file.is_file():
-  sys.exit()
+platforms = load_yaml('/opt/netbox/initializers/platforms.yml')
 
-with file.open('r') as stream:
-  yaml = YAML(typ='safe')
-  platforms = yaml.load(stream)
+if platforms is None:
+  sys.exit()
 
 optional_assocs = {
   'manufacturer': (Manufacturer, 'name'),
 }
 
-  if platforms is not None:
 for params in platforms:
 
   for assoc, details in optional_assocs.items():

startup_scripts/110_tenant_groups.py

@@ -1,17 +1,12 @@
 from tenancy.models import TenantGroup
-from ruamel.yaml import YAML
-from pathlib import Path
+from startup_script_utils import load_yaml
 import sys
 
-file = Path('/opt/netbox/initializers/tenant_groups.yml')
-if not file.is_file():
+tenant_groups = load_yaml('/opt/netbox/initializers/tenant_groups.yml')
+
+if tenant_groups is None:
   sys.exit()
 
-with file.open('r') as stream:
-  yaml = YAML(typ='safe')
-  tenant_groups = yaml.load(stream)
-
-  if tenant_groups is not None:
 for params in tenant_groups:
   tenant_group, created = TenantGroup.objects.get_or_create(**params)
 

startup_scripts/120_tenants.py

@@ -1,23 +1,17 @@
 from tenancy.models import Tenant, TenantGroup
 from extras.models import CustomField, CustomFieldValue
-from ruamel.yaml import YAML
-
-from pathlib import Path
+from startup_script_utils import load_yaml
 import sys
 
-file = Path('/opt/netbox/initializers/tenants.yml')
-if not file.is_file():
-  sys.exit()
+tenants = load_yaml('/opt/netbox/initializers/tenants.yml')
 
-with file.open('r') as stream:
-  yaml = YAML(typ='safe')
-  tenants = yaml.load(stream)
+if tenants is None:
+  sys.exit()
 
 optional_assocs = {
   'group': (TenantGroup, 'name')
 }
 
-  if tenants is not None:
 for params in tenants:
   custom_fields = params.pop('custom_fields', None)
 

startup_scripts/130_devices.py

@@ -3,18 +3,13 @@ from ipam.models import IPAddress
 from virtualization.models import Cluster
 from tenancy.models import Tenant
 from extras.models import CustomField, CustomFieldValue
-from ruamel.yaml import YAML
-
-from pathlib import Path
+from startup_script_utils import load_yaml
 import sys
 
-file = Path('/opt/netbox/initializers/devices.yml')
-if not file.is_file():
-  sys.exit()
+devices = load_yaml('/opt/netbox/initializers/devices.yml')
 
-with file.open('r') as stream:
-  yaml = YAML(typ='safe')
-  devices = yaml.load(stream)
+if devices is None:
+  sys.exit()
 
 required_assocs = {
   'device_role': (DeviceRole, 'name'),
@@ -31,7 +26,6 @@ with file.open('r') as stream:
   'primary_ip6': (IPAddress, 'address')
 }
 
-  if devices is not None:
 for params in devices:
   custom_fields = params.pop('custom_fields', None)
 

startup_scripts/140_cluster_types.py

@@ -1,17 +1,12 @@
 from virtualization.models import ClusterType
-from ruamel.yaml import YAML
-from pathlib import Path
+from startup_script_utils import load_yaml
 import sys
 
-file = Path('/opt/netbox/initializers/cluster_types.yml')
-if not file.is_file():
+cluster_types = load_yaml('/opt/netbox/initializers/cluster_types.yml')
+
+if cluster_types is None:
   sys.exit()
 
-with file.open('r') as stream:
-  yaml = YAML(typ='safe')
-  cluster_types = yaml.load(stream)
-
-  if cluster_types is not None:
 for params in cluster_types:
   cluster_type, created = ClusterType.objects.get_or_create(**params)
 

startup_scripts/150_rirs.py

@@ -1,17 +1,12 @@
 from ipam.models import RIR
-from ruamel.yaml import YAML
-from pathlib import Path
+from startup_script_utils import load_yaml
 import sys
 
-file = Path('/opt/netbox/initializers/rirs.yml')
-if not file.is_file():
+rirs = load_yaml('/opt/netbox/initializers/rirs.yml')
+
+if rirs is None:
   sys.exit()
 
-with file.open('r') as stream:
-  yaml = YAML(typ='safe')
-  rirs = yaml.load(stream)
-
-  if rirs is not None:
 for params in rirs:
   rir, created = RIR.objects.get_or_create(**params)
 

startup_scripts/160_aggregates.py

@@ -1,24 +1,20 @@
 from ipam.models import Aggregate, RIR
-from ruamel.yaml import YAML
+
 from extras.models import CustomField, CustomFieldValue
 
 from netaddr import IPNetwork
-from pathlib import Path
+from startup_script_utils import load_yaml
 import sys
 
-file = Path('/opt/netbox/initializers/aggregates.yml')
-if not file.is_file():
-  sys.exit()
+aggregates = load_yaml('/opt/netbox/initializers/aggregates.yml')
 
-with file.open('r') as stream:
-  yaml = YAML(typ='safe')
-  aggregates = yaml.load(stream)
+if aggregates is None:
+  sys.exit()
 
 required_assocs = {
   'rir': (RIR, 'name')
 }
 
-  if aggregates is not None:
 for params in aggregates:
   custom_fields = params.pop('custom_fields', None)
   params['prefix'] = IPNetwork(params['prefix'])

startup_scripts/170_clusters.py

@@ -1,18 +1,13 @@
 from dcim.models import Site
 from virtualization.models import Cluster, ClusterType, ClusterGroup
 from extras.models import CustomField, CustomFieldValue
-from ruamel.yaml import YAML
-
-from pathlib import Path
+from startup_script_utils import load_yaml
 import sys
 
-file = Path('/opt/netbox/initializers/clusters.yml')
-if not file.is_file():
-  sys.exit()
+clusters = load_yaml('/opt/netbox/initializers/clusters.yml')
 
-with file.open('r') as stream:
-  yaml = YAML(typ='safe')
-  clusters = yaml.load(stream)
+if clusters is None:
+  sys.exit()
 
 required_assocs = {
   'type': (ClusterType, 'name')
@@ -23,7 +18,6 @@ with file.open('r') as stream:
   'group': (ClusterGroup, 'name')
 }
 
-  if clusters is not None:
 for params in clusters:
   custom_fields = params.pop('custom_fields', None)
 

startup_scripts/180_vrfs.py

@@ -1,24 +1,20 @@
 from ipam.models import VRF
 from tenancy.models import Tenant
-from ruamel.yaml import YAML
+
 from extras.models import CustomField, CustomFieldValue
 
-from pathlib import Path
+from startup_script_utils import load_yaml
 import sys
 
-file = Path('/opt/netbox/initializers/vrfs.yml')
-if not file.is_file():
-  sys.exit()
+vrfs = load_yaml('/opt/netbox/initializers/vrfs.yml')
 
-with file.open('r') as stream:
-  yaml = YAML(typ='safe')
-  vrfs = yaml.load(stream)
+if vrfs is None:
+  sys.exit()
 
 optional_assocs = {
   'tenant': (Tenant, 'name')
 }
 
-  if vrfs is not None:
 for params in vrfs:
   custom_fields = params.pop('custom_fields', None)
 

startup_scripts/190_prefix_vlan_roles.py

@@ -1,17 +1,12 @@
 from ipam.models import Role
-from ruamel.yaml import YAML
-from pathlib import Path
+from startup_script_utils import load_yaml
 import sys
 
-file = Path('/opt/netbox/initializers/prefix_vlan_roles.yml')
-if not file.is_file():
+roles = load_yaml('/opt/netbox/initializers/prefix_vlan_roles.yml')
+
+if roles is None:
   sys.exit()
 
-with file.open('r') as stream:
-  yaml = YAML(typ='safe')
-  roles = yaml.load(stream)
-
-  if roles is not None:
 for params in roles:
   role, created = Role.objects.get_or_create(**params)
 

startup_scripts/200_vlan_groups.py

@@ -1,24 +1,18 @@
 from dcim.models import Site
 from ipam.models import VLANGroup
 from extras.models import CustomField, CustomFieldValue
-from ruamel.yaml import YAML
-
-from pathlib import Path
+from startup_script_utils import load_yaml
 import sys
 
-file = Path('/opt/netbox/initializers/vlan_groups.yml')
-if not file.is_file():
-  sys.exit()
+vlan_groups = load_yaml('/opt/netbox/initializers/vlan_groups.yml')
 
-with file.open('r') as stream:
-  yaml = YAML(typ='safe')
-  vlan_groups = yaml.load(stream)
+if vlan_groups is None:
+  sys.exit()
 
 optional_assocs = {
   'site': (Site, 'name')
 }
 
-  if vlan_groups is not None:
 for params in vlan_groups:
   custom_fields = params.pop('custom_fields', None)
 

startup_scripts/210_vlans.py

@@ -2,18 +2,13 @@ from dcim.models import Site
 from ipam.models import VLAN, VLANGroup, Role
 from tenancy.models import Tenant, TenantGroup
 from extras.models import CustomField, CustomFieldValue
-from ruamel.yaml import YAML
-
-from pathlib import Path
+from startup_script_utils import load_yaml
 import sys
 
-file = Path('/opt/netbox/initializers/vlans.yml')
-if not file.is_file():
-  sys.exit()
+vlans = load_yaml('/opt/netbox/initializers/vlans.yml')
 
-with file.open('r') as stream:
-  yaml = YAML(typ='safe')
-  vlans = yaml.load(stream)
+if vlans is None:
+  sys.exit()
 
 optional_assocs = {
   'site': (Site, 'name'),
@@ -23,7 +18,6 @@ with file.open('r') as stream:
   'role': (Role, 'name')
 }
 
-  if vlans is not None:
 for params in vlans:
   custom_fields = params.pop('custom_fields', None)
 

startup_scripts/220_prefixes.py

@@ -2,19 +2,14 @@ from dcim.models import Site
 from ipam.models import Prefix, VLAN, Role, VRF
 from tenancy.models import Tenant, TenantGroup
 from extras.models import CustomField, CustomFieldValue
-from ruamel.yaml import YAML
-
 from netaddr import IPNetwork
-from pathlib import Path
+from startup_script_utils import load_yaml
 import sys
 
-file = Path('/opt/netbox/initializers/prefixes.yml')
-if not file.is_file():
-  sys.exit()
+prefixes = load_yaml('/opt/netbox/initializers/prefixes.yml')
 
-with file.open('r') as stream:
-  yaml = YAML(typ='safe')
-  prefixes = yaml.load(stream)
+if prefixes is None:
+  sys.exit()
 
 optional_assocs = {
   'site': (Site, 'name'),
@@ -25,7 +20,6 @@ with file.open('r') as stream:
   'vrf': (VRF, 'name')
 }
 
-  if prefixes is not None:
 for params in prefixes:
   custom_fields = params.pop('custom_fields', None)
   params['prefix'] = IPNetwork(params['prefix'])
@@ -34,7 +28,6 @@ with file.open('r') as stream:
     if assoc in params:
       model, field = details
       query = { field: params.pop(assoc) }
-
       params[assoc] = model.objects.get(**query)
 
   prefix, created = Prefix.objects.get_or_create(**params)
@@ -48,7 +41,6 @@ with file.open('r') as stream:
           obj=prefix,
           value=cf_value
         )
-
         prefix.custom_field_values.add(custom_field_value)
 
     print("📌 Created Prefix", prefix.prefix)

startup_scripts/230_virtual_machines.py

@@ -2,18 +2,13 @@ from dcim.models import Site, Platform, DeviceRole
 from virtualization.models import Cluster, VirtualMachine
 from tenancy.models import Tenant
 from extras.models import CustomField, CustomFieldValue
-from ruamel.yaml import YAML
-
-from pathlib import Path
+from startup_script_utils import load_yaml
 import sys
 
-file = Path('/opt/netbox/initializers/virtual_machines.yml')
-if not file.is_file():
-  sys.exit()
+virtual_machines = load_yaml('/opt/netbox/initializers/virtual_machines.yml')
 
-with file.open('r') as stream:
-  yaml = YAML(typ='safe')
-  virtual_machines = yaml.load(stream)
+if virtual_machines is None:
+  sys.exit()
 
 required_assocs = {
   'cluster': (Cluster, 'name')
@@ -25,7 +20,6 @@ with file.open('r') as stream:
   'role': (DeviceRole, 'name')
 }
 
-  if virtual_machines is not None:
 for params in virtual_machines:
   custom_fields = params.pop('custom_fields', None)
 

startup_scripts/240_virtualization_interfaces.py

@@ -1,24 +1,18 @@
 from dcim.models import Interface
 from virtualization.models import VirtualMachine
 from extras.models import CustomField, CustomFieldValue
-from ruamel.yaml import YAML
-
-from pathlib import Path
+from startup_script_utils import load_yaml
 import sys
 
-file = Path('/opt/netbox/initializers/virtualization_interfaces.yml')
-if not file.is_file():
-  sys.exit()
+interfaces = load_yaml('/opt/netbox/initializers/virtualization_interfaces.yml')
 
-with file.open('r') as stream:
-  yaml = YAML(typ='safe')
-  interfaces = yaml.load(stream)
+if interfaces is None:
+  sys.exit()
 
 required_assocs = {
   'virtual_machine': (VirtualMachine, 'name')
 }
 
-  if interfaces is not None:
 for params in interfaces:
   custom_fields = params.pop('custom_fields', None)
 

startup_scripts/250_dcim_interfaces.py

@@ -1,23 +1,17 @@
 from dcim.models import Interface, Device
 from extras.models import CustomField, CustomFieldValue
-from ruamel.yaml import YAML
-
-from pathlib import Path
+from startup_script_utils import load_yaml
 import sys
 
-file = Path('/opt/netbox/initializers/dcim_interfaces.yml')
-if not file.is_file():
-  sys.exit()
+interfaces= load_yaml('/opt/netbox/initializers/dcim_interfaces.yml')
 
-with file.open('r') as stream:
-  yaml = YAML(typ='safe')
-  interfaces = yaml.load(stream)
+if interfaces is None:
+  sys.exit()
 
 required_assocs = {
   'device': (Device, 'name')
 }
 
-  if interfaces is not None:
 for params in interfaces:
   custom_fields = params.pop('custom_fields', None)
 

startup_scripts/260_ip_addresses.py

@@ -3,19 +3,15 @@ from dcim.models import Device, Interface
 from virtualization.models import VirtualMachine
 from tenancy.models import Tenant
 from extras.models import CustomField, CustomFieldValue
-from ruamel.yaml import YAML
 
 from netaddr import IPNetwork
-from pathlib import Path
+from startup_script_utils import load_yaml
 import sys
 
-file = Path('/opt/netbox/initializers/ip_addresses.yml')
-if not file.is_file():
-  sys.exit()
+ip_addresses = load_yaml('/opt/netbox/initializers/ip_addresses.yml')
 
-with file.open('r') as stream:
-  yaml = YAML(typ='safe')
-  ip_addresses = yaml.load(stream)
+if ip_addresses is None:
+  sys.exit()
 
 optional_assocs = {
   'tenant': (Tenant, 'name'),
@@ -23,7 +19,6 @@ with file.open('r') as stream:
   'interface': (Interface, 'name')
 }
 
-  if ip_addresses is not None:
 for params in ip_addresses:
   vm = params.pop('virtual_machine', None)
   device = params.pop('device', None)

startup_scripts/startup_script_utils/__init__.py

@@ -0,0 +1,2 @@
+from .load_yaml import load_yaml
+from .permissions import set_permissions

startup_scripts/startup_script_utils/load_yaml.py

@@ -0,0 +1,10 @@
+from ruamel.yaml import YAML
+from pathlib import Path
+
+def load_yaml(yaml_file: str):
+  yf = Path(yaml_file)
+  if not yf.is_file():
+    return None
+  with yf.open("r") as stream:
+    yaml = YAML(typ="safe")
+    return yaml.load(stream)

startup_scripts/startup_script_utils/permissions.py

@@ -0,0 +1,18 @@
+from django.contrib.auth.models import Permission
+
+
+def set_permissions(subject, permission_filters):
+  if subject is None or permission_filters is None:
+    return
+  subject.clear()
+  for permission_filter in permission_filters:
+    if "*" in permission_filter:
+      permission_filter_regex = "^" + permission_filter.replace("*", ".*") + "$"
+      permissions = Permission.objects.filter(codename__iregex=permission_filter_regex)
+      print("  ⚿ Granting", permissions.count(), "permissions matching '" + permission_filter + "'")
+    else:
+      permissions = Permission.objects.filter(codename=permission_filter)
+      print("  ⚿ Granting permission", permission_filter)
+
+    for permission in permissions:
+      subject.add(permission)