Meshviewer / Knotenkarte

Es gibt nun 2 Karten, eine für Pinneberg, und eine für Helgoland Die Karten laufen derzeit auf gate07.

Das frontend liegt jeweils in /var/www/vhosts/meshviewer.pinneberg.freifunk.net und /var/www/vhosts/meshviewer.helgoland.freifunk.net

Der Quellcode stammt von https://github.com/freifunk/meshviewer Im Unterverzeichnis device-pictures liegt das Repository https://github.com/freifunk/device-pictures um Grafiken der Router anzeigen zu können. Alle Einstellungen der Karte liegen in config.json.

Die Daten der Knoten kommen nun nicht mehr per alfred sondern per respondd. Auf den Knoten lief das tool schon immer mit, auf den Gateways musste es nachinstalliert werden. Um die respondd daten in ein für den meshviewer lesbares format zu bringen wird nun das tool yanic benutzt. Quellcode zu finden unter https://github.com/FreifunkBremen/yanic yanic läuft auf gate07 als deamon im Hintergrund und liefert die daten für beide Karten.

Die yanic Konfiguration ist das aufwändigste an dem ganze.

/etc/yanic.conf

# This is the config file for Yanic written in "Tom's Obvious, Minimal Language."
# syntax: https://github.com/toml-lang/toml
# (if you need something multiple times, checkout out the [[array of table]] section)
 
# Send respondd request to update information
[respondd]
enable           = true
# Delay startup until a multiple of the period since zero time
synchronize      = "1m"
# how often request per multicast
collect_interval = "1m"
 
# If you have custom respondd fields, you can ask Yanic to also collect these.
# NOTE: This does not automatically include these fields in the output.
#       The meshviewer-ffrgb output module will include them under "custom_fields",
#       but other modules may simply ignore them.
#[[respondd.custom_field]]
#name = zip
# You can use arbitrary GJSON expressions here, see https://github.com/tidwall/gjson
# We expect this expression to return a string.
#path = nodeinfo.location.zip
 
# table of a site to save stats for (not exists for global only)
#[respondd.sites.example]
## list of domains on this site to save stats for (empty for global only)
#domains = []
## example
#[respondd.sites.ffhb]
#domains            = ["city"]
 
# interface that has an IP in your mesh network
[[respondd.interfaces]]
# name of interface on which this collector is running
ifname = "br-ffpi"
# ip address which is used for sending
# (optional - without definition used a address of ifname - preferred link local)
#ip_address = "fd2f:5119:f2d::5"
# disable sending multicast respondd request
# (for receiving only respondd packages e.g. database respondd)
#send_no_request = false
# multicast address to destination of respondd
# (optional - without definition used default ff05::2:1001)
# Very old gluon uses "ff02::2:1001" as multicast, newer use ff05::2:1001. If you have old and new
# gluon nodes on the same network, create a separate "respondd.interfaces" section for each mutlicast address.
#multicast_address = "ff02::2:1001"
# define a port to listen
# if not set or set to 0 the kernel will use a random free port at its own
#port = 10001
 
# A little build-in webserver, which statically serves a directory.
# This is useful for testing purposes or for a little standalone installation.
[webserver]
enable  = false
bind    = "127.0.0.1:8080"
webroot = "/var/www/vhosts/meshviewer.pinneberg.freifunk.net"
 
 
[nodes]
# Cache file
# a json file to cache all data collected directly from respondd
state_path    = "/var/lib/yanic/state.json"
# prune data in RAM, cache-file and output json files (i.e. nodes.json)
# that were inactive for longer than
prune_after   = "36500d"
# Export nodes and graph periodically
save_interval = "5s"
# Set node to offline if not seen within this period
offline_after = "10m"
 
no_owner = false
 
## [[nodes.output.example]]
# Each output format has its own config block and needs to be enabled by adding:
#enable = true
#
# For each output format there can be set different filters
#[nodes.output.example.filter]
#
# WARNING: if it is not set, it will publish contact information of other persons
# Set to true, if you did not want the json files to contain the owner information
#no_owner = false
#
# List of nodeids of nodes that should be filtered out, so they won't appear in output
#blocklist = ["00112233445566", "1337f0badead"]
#
# List of site_codes of nodes that should be included in the output
#sites = ["ffhb"]
#
# replace the site_code with the domain_code in this output
# e.g. site_code='ffhb',domain_code='city' => site_code='city', domain_code=''
#domain_as_site = true
#
# append on the site_code the domain_code with a '.' in this output
# e.g. site_code='ffhb',domain_code='city' => site_code='ffhb.city', domain_code=''
#domain_append_site = true
#
# set has_location to true if you want to include only nodes that have geo-coordinates set
# (setting this to false has no sensible effect, unless you'd want to hide nodes that have coordinates)
#has_location = true
 
#[nodes.output.example.filter.in_area]
# nodes outside this area are not shown on the map but are still listed as a node without coordinates
#latitude_min = 34.30
#latitude_max = 71.85
#longitude_min = -24.96
#longitude_max = 39.72
 
 
# outputs all nodes as points into nodes.geojson
[[nodes.output.geojson]]
enable   = true
path = "/var/www/vhosts/meshviewer.pinneberg.freifunk.net/data.global/nodes.geojson"
 
# definition for the new more compressed meshviewer.json
[[nodes.output.meshviewer-ffrgb]]
enable   = true
path = "/var/www/vhosts/meshviewer.pinneberg.freifunk.net/data.global/meshviewer.json"
 
# definition for nodes.json
[[nodes.output.meshviewer]]
enable = true
# The structure version of the output which should be generated (i.e. nodes.json)
# version 1 is accepted by the legacy meshviewer (which is the master branch)
#   i.e. https://github.com/ffnord/meshviewer/tree/master
# version 2 is accepted by the new versions of meshviewer (which are in the legacy develop branch or newer)
#   i.e. https://github.com/ffnord/meshviewer/tree/dev
#        https://github.com/ffrgb/meshviewer/tree/develop
version    = 2
# path where to store nodes.json
nodes_path = "/var/www/vhosts/meshviewer.pinneberg.freifunk.net/data.global/nodes.json"
# path where to store graph.json
graph_path = "/var/www/vhosts/meshviewer.pinneberg.freifunk.net/data.global/graph.json"
 
no_owner = false
 
#[nodes.output.meshviewer.filter]
# WARNING: if it is not set, it will publish contact information of other persons
#no_owner = false
 
 
# definition for nodelist.json
[[nodes.output.nodelist]]
enable   = true
path = "/var/www/vhosts/meshviewer.pinneberg.freifunk.net/data.global/nodelist.json"
 
#[nodes.output.nodelist.filter]
# WARNING: if it is not set, it will publish contact information of other persons
#no_owner = false
 
# definition for prometheus-sd.json
[[nodes.output.prometheus-sd]]
enable  = true
path    = "/var/www/vhosts/meshviewer.pinneberg.freifunk.net/data.global/prometheus-sd.json"
# ip = lates recieved ip, node_id = node id from host
target_address = "ip"
 
# Labels of the data (optional)
[nodes.output.prometheus-sd.labels]
#labelname1 = "labelvalue 1"
## some useful e.g.:
#hosts   = "ffhb"
#service = "yanic"
 
# definition for raw.json
[[nodes.output.raw]]
enable   = true
path = "/var/www/vhosts/meshviewer.pinneberg.freifunk.net/data.global/raw.json"
 
#[nodes.output.raw.filter]
# WARNING: if it is not set, it will publish contact information of other persons
#no_owner = false
 
 
################
# PINNEBERG
# ##############
[[nodes.output.meshviewer]]
enable = true
version = 2
nodes_path = "/var/www/vhosts/meshviewer.pinneberg.freifunk.net/data/nodes.json"
graph_path = "/var/www/vhosts/meshviewer.pinneberg.freifunk.net/data/graph.json"
no_owner = false
domains = ["ffpi_ibss", "ffpi_80211s", "ffpi"]
 
[nodes.output.meshviewer.filter]
# Filtern nach den spezifischen Domains
domain_append_site = true
sites = ["ffpi.ffpi_ibss", "ffpi.ffpi_80211s", ".ffpi"]
 
 
[[nodes.output.meshviewer-ffrgb]]
enable = true
no_owner = false
path = "/var/www/vhosts/meshviewer.pinneberg.freifunk.net/data/meshviewer.json"
domains = ["ffpi_ibss", "ffpi_80211s", "ffpi"]
 
[nodes.output.meshviewer-ffrgb.filter]
# Filtern nach den spezifischen Domains
domain_append_site = true
sites = ["ffpi.ffpi_ibss", "ffpi.ffpi_80211s", ".ffpi"]
 
 
[[nodes.output.geojson]]
enable = true
path = "/var/www/vhosts/meshviewer.pinneberg.freifunk.net/data/nodes.geojson"
domains = ["ffpi_ibss", "ffpi_80211s", "ffpi"]
 
[nodes.output.geojson.filter]
# Filtern nach den spezifischen Domains
domain_append_site = true
sites = ["ffpi.ffpi_ibss", "ffpi.ffpi_80211s", ".ffpi"]
 
 
# definition for nodelist.json
[[nodes.output.nodelist]]
enable   = true
path = "/var/www/vhosts/meshviewer.pinneberg.freifunk.net/data/nodelist.json"
domains = ["ffpi_ibss", "ffpi_80211s", "ffpi"]
 
[nodes.output.nodelist.filter]
domain_append_site = true
sites = ["ffpi.ffpi_ibss", "ffpi.ffpi_80211s", ".ffpi"]
 
 
[[nodes.output.raw]]
enable = true
path = "/var/www/vhosts/meshviewer.pinneberg.freifunk.net/data/raw.json"
domains = ["ffpi_ibss", "ffpi_80211s", "ffpi"]
 
[nodes.output.raw.filter]
# Filtern nach den spezifischen Domains
domain_append_site = true
sites = ["ffpi.ffpi_ibss", "ffpi.ffpi_80211s", ".ffpi"]
 
 
###############
# HELGOLAND
###############
[[nodes.output.meshviewer]]
enable = true
version = 2
nodes_path = "/var/www/vhosts/meshviewer.helgoland.freifunk.net/data/nodes.json"
graph_path = "/var/www/vhosts/meshviewer.helgoland.freifunk.net/data/graph.json"
no_owner = false
 
[nodes.output.meshviewer.filter]
# Filtern nach den spezifischen Domains
domain_append_site = true
sites = ["ffpi.helgoland", "ffpi.helgoland_old", ".ffpi", "helgo.helgoland_old"]
 
 
[[nodes.output.meshviewer-ffrgb]]
enable = true
no_owner = false
path = "/var/www/vhosts/meshviewer.helgoland.freifunk.net/data/meshviewer.json"
 
[nodes.output.meshviewer-ffrgb.filter]
# Filtern nach den spezifischen Domains
domain_append_site = true
sites = ["ffpi.helgoland", "ffpi.helgoland_old", ".ffpi", "helgo.helgoland_old"]
 
 
[[nodes.output.geojson]]
enable = true
path = "/var/www/vhosts/meshviewer.helgoland.freifunk.net/data/nodes.geojson"
 
[nodes.output.geojson.filter]
# Filtern nach den spezifischen Domains
domain_append_site = true
sites = ["ffpi.helgoland", "ffpi.helgoland_old", ".ffpi", "helgo.helgoland_old"]
 
 
[[nodes.output.nodelist]]
enable   = true
path = "/var/www/vhosts/meshviewer.helgoland.freifunk.net/data/nodelist.json"
 
[nodes.output.nodelist.filter]
domain_append_site = true
sites = ["ffpi.helgoland", "ffpi.helgoland_old", ".ffpi", "helgo.helgoland_old"]
 
 
[[nodes.output.raw]]
enable = true
path = "/var/www/vhosts/meshviewer.helgoland.freifunk.net/data/raw.json"
 
[nodes.output.raw.filter]
# Filtern nach den spezifischen Domains
sites = ["ffpi.helgoland", "ffpi.helgoland_old", ".ffpi", "helgo.helgoland_old"]
 
 
 
 
 
[database]
# this will send delete commands to the database to prune data
# which is older than:
delete_after    = "365d"
# how often run the cleaning
delete_interval = "1h"
 
## [[database.connection.example]]
# Each database-connection has its own config block and needs to be enabled by adding:
#enable = true
 
# Save collected data to InfluxDB.
# There are the following measurements:
#   node: store node specific data i.e. clients memory, airtime
#   link: store link tq between two interfaces of two different nodes
#   global: store global data, i.e. count of clients and nodes
#   firmware: store the count of nodes tagged with firmware
#   model: store the count of nodes tagged with hardware model
#   autoupdater: store the count of autoupdate branch
[[database.connection.influxdb]]
enable   = false
address  = "http://localhost:8086"
database = "ffhb"
username = ""
password = ""
#insecure_skip_verify = true
 
# Tagging of the data (optional)
[database.connection.influxdb.tags]
# Tags used by Yanic would override the tags from this config
# nodeid, hostname, owner, model, firmware_base, firmware_release,frequency11g and frequency11a are tags which are already used
#tagname1 = "tagvalue 1"
# some useful e.g.:
#system   = "productive"
#site     = "ffhb"
 
# Save collected data to InfluxDB2.
# There are the following measurements:
#   node: store node specific data i.e. clients memory, airtime
#   link: store link tq between two interfaces of two different nodes with i.e. nodeid, address, hostname
#   global: store global data, i.e. count of clients and nodes
#   firmware: store the count of nodes tagged with firmware
#   model: store the count of nodes tagged with hardware model
#   autoupdater: store the count of autoupdate branch
[[database.connection.influxdb2]]
enable   = false
address  = "http://localhost:8086"
token = ""
organization_id = ""
# fallback of no specific bucket for measurement is setup
bucket_default = "yanic"
 
# Specify bucket per measurement (of not set fallback bucket_default is used or panic)
#
# WARNING:
#   yanic does NOT prune node's data (so please set up data retention in InfluxDB2 setup).
#
#   We highly recommend to setup e.g. Data retention in your InfluxDB2 server per measurements.
#   https://docs.influxdata.com/influxdb/v2/reference/internals/data-retention/
#
[database.connection.influxdb2.buckets]
#link = "yanic-temp"
#node = "yanic-temp"
#dhcp = "yanic-temp"
global = "yanic-persistent"
#firmware = "yanic-temp"
#model = "yanic-temp"
#autoupdater = "yanic-temp"
 
# Tagging of the data (optional)
[database.connection.influxdb2.tags]
# Tags used by Yanic would override the tags from this config
# nodeid, hostname, owner, model, firmware_base, firmware_release,frequency11g and frequency11a are tags which are already used
#tagname1 = "tagvalue 1"
# some useful e.g.:
#system   = "productive"
#site     = "ffhb"
 
# Graphite settings
[[database.connection.graphite]]
enable   = false
address  = "localhost:2003"
# Graphite is replacing every "." in the metric name with a slash "/" and uses
# that for the file system hierarchy it generates. it is recommended to at least
# move the metrics out of the root namespace (that would be the empty prefix).
# If you only intend to run one community and only freifunk on your graphite node
# then the prefix can be set to anything (including the empty string) since you
# probably wont care much about "polluting" the namespace.
prefix   = "freifunk"
 
# respondd (yanic)
# forward collected respondd package to a address
# (e.g. to another respondd collector like a central yanic instance or hopglass)
[[database.connection.respondd]]
enable   = false
# type of network to create a connection
type     = "udp6"
# destination address to connect/send respondd package
address  = "stats.bremen.freifunk.net:11001"
 
# Logging
[[database.connection.logging]]
enable   = false
path     = "/var/log/yanic.log"

Die Karte läuft momentan auf dem Server, der auch gate01 fährt. Um die notwendigen Daten zu sammeln muß das Backend ffmap-backend auf die Programme alfred und batadv-vis zurückgreifen. Folglich muß das Backendprogramm auf einem Rechner laufen, der Teil des Meshs ist. Das Backend schreibt die Daten in drei JSON-Dateien, welche sinnvollerweise in einem Datenverzeichnis abgelegt werden. In den Dateien ist beispielsweise das erste Auftauchen eines Knotens vermerkt:

• nodelist.json - Kurze Knotenliste
• nodes.json - Detaildaten zu den einzelnen Knoten
• graph.json - Beschreibung der Verbindungen der Knoten untereinander
• blacklist.json - Korrekturmöglichkeit für Knoten, die die Karte „sprengen“

Werden die Dateien gelöscht, beginnt man ganz von vorne, d.h. alle Knoten werden wieder als neu angesehen.

Hinter den Knoten werden ggf. verschiedene Symbole angezeigt:

Symbol	Bedeutung
	Koordinaten sind vorhanden, Knoten wird auf der Karte angezeigt
	Mobiler Knoten (role=mobile)
	Temporärer Knoten z.B. nur tagsüber verfügbar (role=temp)
	VPN Offloader (role=offloader)
	Testknoten (role=test)
	Gateway
	Knoten für Dienste (Server)

Das Backend heißt ffmap-backend, das Pinneberger Programm ist zu finden unter: https://git.pinneberg.freifunk.net/thooge/ffmap-backend

Abhängigkeiten von Programmen

• batctl
• alfred-json
• batadv-vis

Voraussetzung für den Betrieb:

aptitude install python3 python3-networkx python-configparser rrdtool

Die Daten werden unter Debian am besten unter /var/lib gespeichert. Wir verwenden:

• /var/lib/ffmap ist das Basisverzeichnis
  • die JSON-Dateien speichern wir in das Verzeichnis /var/lib/ffmap/mapdata, dieses wird über die Konfiguration des virtuellen Servers als Alias in den Webspace des Meshviewers eingebunden
  • die RRD-Dateien speichern wir in das Verzeichnis /var/lib/ffmap/nodedb
• Konfigurationsdateien gehören nach /etc, wir verwenden /etc/ffmap:
  • aliases.json
  • ffmap.cfg

Es wird eine Protokolldatei geschrieben. Als Standard wird /var/log/ffmap.log verwendet. Der Umfang der Protokollierung kann über die Konfigurationsdatei mittels loglevel eingestellt werden. Die Protokollierung kann durch einen Loglevel von 0 ausgestellt werden.

Die Datenaktualisierung erfolgt jede Minute über einen Cron-job:

/etc/cron.d/ffmap

# Meshviewer backend
 
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
 
*/1 * * * * root /usr/local/bin/ffmap-pi/backend.py -d /var/lib/ffmap/mapdata --with-rrd

Beispiel für eine Konfigurationsdatei des Backends:

/etc/ffmap/ffmap.cfg

# configuration for freifunk map utilities
 
[global]
logfile = /var/log/ffmap.log
loglevel = 2
dest_dir = /var/lib/ffmap/mapdata
aliases = /etc/ffmap/aliases.json
prune = 180
 
[rrd]
enabled = yes
graphs = no
nodedb = /var/lib/ffmap/nodedb

Wenn protokolliert wird, kann das eine ganze Menge an Daten ergeben, weil das Programm im Normalfall jede Minute ausgeführt wird. Da ist es sinnvoll, die Rotation der Logdateien zu konfigurieren:

/etc/logrotate.d/ffmap

/var/log/ffmap.log {
    daily 
    rotate 7 
    compress
    delaycompress
    notifempty
    missingok
}

Wir benutzen die Version 4 des Frontends mit einigen kleinen eigenen Änderungen. Die Quellen befinden sich unter: https://github.com/ffnord/meshviewer

aptitude install nodejs nodejs-legacy npm ruby-sass

git clone https://github.com/ffnord/meshviewer.git
cd meshviewer
git checkout v4
npm install
npm install bower grunt-cli
node_modules/.bin/bower install
ln -s /var/lib/ffmap/mapdata data

Unter Wheezy muß die Datei scss/main.scss angepaßt werden.

/* @media screen and (max-width: $minscreenwidth) { */
@media screen and (max-width: 630pt) { 

Nach einer Änderung in der Konfiguration muß der folgende Befehl aufgerufen werden:

node_modules/.bin/grunt

cp config.json.example config.json

Config.json bearbeiten

RESTORE

Wenn es keinen Knoten mit Koordinaten gibt, dann wird die Leaflet-Karte nicht angezeigt

Ohne den ganzen NodeJS-Plunder. Es werden soweit es geht Pakete des Betriebssystems verwendet, Siehe dazu auch den Helgoländer Meshviewer.

apt-get install libjs-moment libjs-d3 libjs-requirejs libjs-es6-shim

aptitude install php-gd php-rrd

Die Statistiken sind im Unterverzeichnis stats zu finden.