Visualization in Software

Prelude

Jonathan Llovet

  • Software Engineer, Linguist, Philosopher

jllovet.com

Jonathan Llovet

School

  • BA from St. John’s College, studied Philosophy and History of Math and Science
  • Enrolled in MS in CS at Towson University
  • Summer Language School in Hebrew at Middlebury University
  • Coursework at Indiana Univ. East in Math
  • Other language programs - classical and modern

jllovet.com

Jonathan Llovet

Work

  • Solutions Architect and Cybersecurity Manager at Inspire Brands
  • Founder and CEO of Opening Questions, where I’m building a language learning app for Ancient Greek called Talk with Albert
  • Formerly worked as a consultant in Identity and Access Management
  • Writing a book on Philosophy of Computer Science: Algorithmics - The Being of Becoming

jllovet.com

Why Talk About Visualization?

What is Computation?

Application of Algorithmics

  • Study of change and becoming, sister of calculus
  • Transformation of initial state to a subsequent state
  • Sometimes with side effects

Modeling and Analogy

  • Exploration (Questions & Discussions)
  • Instrumentalization (Means to an End)
  • Commodification (Product)

At their core:

  • Communication and Collaboration

We should be concerned with:

  • Rhetoric
  • Effective Presentation

Our Audiences

  • The Computer
  • Other Software Engineers
  • Colleagues
  • Customers
  • Ourselves

Effective Communication

  • Fundamental aspects of our cognition:
    • Space and Time

  • Frame communication around these

    • Allow audience to explore concepts spatially
    • Tell a story

Enter, Graphviz

Hello, Graphviz!

digraph g {
  a -> b
}

g a a b b a->b

  • Declarative graph-generation
  • Plain text files ending in .dot
  • Processed by CLI or another tool

Hello, Graphviz!

digraph g {
  rankdir=LR
  a -> b
}

g a a b b a->b

  • Easy modification and iteration
  • Effective for collaboration
  • VSCode Plugin for live updates

Graphviz Resources

Basic Usage

mygraph.dot

digraph g {
  a -> b
}

Generate a PNG

dot -Tpng mygraph.dot -o mygraph.png

Scale the PNG

dot -Tpng mygraph.dot -s=10 -o mygraph.png

Export to SVG

dot -Tsvg mygraph.dot -o mygraph.svg

Nodes

digraph g {
  rankdir=LR
  // default label is variable name
  seed // default shape is ellipse
  flower [label="Rose"]
  acorn [label="Acorn" shape="square"]
  tree [label="Tree" shape="rectangle"]
  forest [label="Forest" shape="circle"]
}

g seed seed flower Rose acorn Acorn tree Tree forest Forest

Edges

digraph g {
  client [label="client"]
  load_balancer [label="load\nbalancer"]
  server [label="server"]
  db [shape="cylinder" label="db"]
  files [shape="folder" label="file\nbucket"]

  client -> load_balancer [label="request"]
  load_balancer -> server [label="route"]
  server -> {
    files
    db
  } [label="request"]
}

g client client load_balancer load balancer client->load_balancer request server server load_balancer->server route db db server->db request files file bucket server->files request

Subgraphs

digraph g {
  rankdir=LR
  subgraph cluster_unary_functions {
    label="Unary Functions"
    negation [label="Negation"]
    identity [label="Identity"]
  }
  subgraph cluster_binary_functions {
    label="Binary Functions"
    and [label="And"]
    or [label="Or"]
  }
}

g cluster_binary_functions Binary Functions cluster_unary_functions Unary Functions negation Negation identity Identity and And or Or

Styling

digraph g {
    rankdir=LR
    node [colorscheme=pastel25]
    praecoquum [
      shape="rectangle" style=filled
      fillcolor="1"
      label="praecoquum\n------\nLate Latin\nearly-ripening apples"
    ]
    praecocia [
      shape="rectangle" style=filled
      fillcolor="1"
      label="praecocia\n------\nLate Latin\nearly-ripening peaches"
    ]
    praekokion [
      shape="rectangle" style=filled
      fillcolor="2"
      label="πραικόκιον\npraekokion\n------\nAncient Greek\napricot"
    ]
    berikokkia [
      shape="rectangle" style=filled
      fillcolor="2"
      label="βερικοκκία\nberikokkia\n------\nByzantine Greek\napricot tree"
    ]
    albarqūq [
      shape="rectangle" style=filled
      fillcolor="3"
      label="الْبَرْقُوق\nal-barqūq\n------\nArabic\nplums"
    ]
    albercoc [
      shape="rectangle" style=filled
      fillcolor="4"
      label="albercoc\n------\nCatalan\napricot"
    ]
    abrecoc [
      shape="rectangle" style=filled
      fillcolor="4"
      label="abrecoc\n------\ndialectical Catalan\napricot"
    ]
    apricum [
      shape="rectangle" style=filled
      fillcolor="1"
      label="apricum\n------\nLatin\nsunny place"
    ]
    abrecock [
      shape="rectangle" style=filled
      fillcolor="5"
      label="abrecock\n------\nEnglish\nobsolete form of apricot"
    ]
    abricot [
      shape="rectangle" style=filled
      fillcolor="4"
      label="abricot\n------\nFrench\napricot"
    ]
    apricock [
      shape="rectangle" style=filled
      fillcolor="5"
      label="apricock\n------\nEnglish\narchaic form of apricot"
    ]
    apricot [
      shape="rectangle" style=filled
      fillcolor="5"
      label="apricot"
    ]

    praecoquum -> praecocia
    praecocia -> praekokion
    praekokion -> berikokkia
    berikokkia -> albarqūq
    albarqūq -> albercoc
    albercoc -> abrecoc [label="dialectical\nvariation"]
    abrecoc -> abrecock
    abrecoc -> abricot
    apricum -> apricock [label="influenced"]
    abrecock -> apricock [label="alteration"]

    abricot -> apricot [label="influenced"]
    apricock -> apricot
}

See: Wiktionary - Apricot

The History of the Apricot

g praecoquum praecoquum ------ Late Latin early-ripening apples praecocia praecocia ------ Late Latin early-ripening peaches praecoquum->praecocia praekokion πραικόκιον praekokion ------ Ancient Greek apricot praecocia->praekokion berikokkia βερικοκκία berikokkia ------ Byzantine Greek apricot tree praekokion->berikokkia albarqūq الْبَرْقُوق al-barqūq ------ Arabic plums berikokkia->albarqūq

The History of the Apricot

g albarqūq الْبَرْقُوق al-barqūq ------ Arabic plums albercoc albercoc ------ Catalan apricot albarqūq->albercoc abrecoc abrecoc ------ dialectical Catalan apricot albercoc->abrecoc dialectical variation abrecock abrecock ------ English obsolete form of apricot abrecoc->abrecock abricot abricot ------ French apricot abrecoc->abricot apricum apricum ------ Latin sunny place apricock apricock ------ English archaic form of apricot apricum->apricock influenced abrecock->apricock alteration apricot apricot abricot->apricot influenced apricock->apricot

The History of the Apricot

Apricot History Map

A Reprise on Rhetoric and Storytelling

  • Visualizations should be appropriate to the subject
  • Use analogies from reading
  • Weak form of Sapir Whorf
  • Majority of world languages
    • (notable exceptions: Hebrew, Arabic)

Structure Narrative

  • Left-to-Right
  • Top-to-Bottom
  • Clockwise

Putting Things Together - Supply Chain Example

digraph G {
    rankdir="LR"
    ratio=auto
    ranksep=auto

    label="Sculpting Supply Store"
    labelloc="t"

    // Nodes
    subgraph cluster_supply_chain {
        label="Supply Chain"
        style=rounded
        style=filled
        pencolor="#AEB6BE"
        bgcolor="#E5F5FD"
        subgraph cluster_raw_materials {
            label="Raw Materials"
            style=rounded
            style=filled
            pencolor="#AEB6BE"
            bgcolor="#EBF3E7"
            mine [label="Mine"]
        }
        subgraph cluster_clay_supply {
            label="Clay"
            style=rounded
            style=filled
            pencolor="#AEB6BE"
            bgcolor="#EBF3E7"
            clay_supplier [label="Clay\nSupplier"]
        }
        subgraph cluster_tool_supply {
            label="Tools"
            style=rounded
            style=filled
            pencolor="#AEB6BE"
            bgcolor="#EBF3E7"
            tool_manufacturer [label="Tool\nManufacturer"]
            tool_supplier [label="Tool\nSupplier"]

            tool_manufacturer -> tool_supplier [label="Provides"]
        }

        mine -> tool_manufacturer [label="Provides"]
    }
    subgraph cluster_store {
        label="Store"
        style=rounded
        style=filled
        pencolor="#AEB6BE"
        bgcolor="#E5F5FD"
        subgraph cluster_inventory {
            label="Inventory"
            style=rounded
            style=filled
            pencolor="#AEB6BE"
            bgcolor="#EBF3E7"
            clay_inventory [label="Clay\nInventory"]
            tool_inventory [label="Tool\nInventory"]
        }
        subgraph cluster_staff {
            label="Staff"
            style=rounded
            style=filled
            pencolor="#AEB6BE"
            bgcolor="#EBF3E7"
            clerk [label="Clerk"]
            general_manager [label="General\nManager"]
            general_manager -> clerk [label="supports"]
        }
    }

    // Edges
    mine -> clay_supplier [label="Provides"]
    clay_supplier -> clay_inventory [label="Restocks"]
    tool_supplier -> tool_inventory [label="Restocks"]

    clerk -> clay_inventory [label="Manages"]
    clerk -> tool_inventory [label="Manages"]

    general_manager -> clay_supplier [label="Coordinates\nwith"]
    general_manager -> tool_supplier [label="Coordinates\nwith"]
}

G Sculpting Supply Store cluster_supply_chain Supply Chain cluster_raw_materials Raw Materials cluster_clay_supply Clay cluster_tool_supply Tools cluster_store Store cluster_inventory Inventory cluster_staff Staff mine Mine clay_supplier Clay Supplier mine->clay_supplier Provides tool_manufacturer Tool Manufacturer mine->tool_manufacturer Provides clay_inventory Clay Inventory clay_supplier->clay_inventory Restocks tool_supplier Tool Supplier tool_manufacturer->tool_supplier Provides tool_inventory Tool Inventory tool_supplier->tool_inventory Restocks clerk Clerk clerk->clay_inventory Manages clerk->tool_inventory Manages general_manager General Manager general_manager->clay_supplier Coordinates with general_manager->tool_supplier Coordinates with general_manager->clerk supports

Orientation Matters

digraph G {
    rankdir="BT"
    ratio=1
    ranksep=auto

    label="Sculpting Supply Store"
    labelloc="t"

    // Nodes
    subgraph cluster_supply_chain {
        label="Supply Chain"
        style=rounded
        style=filled
        pencolor="#AEB6BE"
        bgcolor="#E5F5FD"
        subgraph cluster_raw_materials {
            label="Raw Materials"
            style=rounded
            style=filled
            pencolor="#AEB6BE"
            bgcolor="#EBF3E7"
            mine [label="Mine"]
        }
        subgraph cluster_clay_supply {
            label="Clay"
            style=rounded
            style=filled
            pencolor="#AEB6BE"
            bgcolor="#EBF3E7"
            clay_supplier [label="Clay\nSupplier"]
        }
        subgraph cluster_tool_supply {
            label="Tools"
            style=rounded
            style=filled
            pencolor="#AEB6BE"
            bgcolor="#EBF3E7"
            tool_manufacturer [label="Tool\nManufacturer"]
            tool_supplier [label="Tool\nSupplier"]

            tool_manufacturer -> tool_supplier [label="Provides"]
        }

        mine -> tool_manufacturer [label="Provides"]
    }
    subgraph cluster_store {
        label="Store"
        style=rounded
        style=filled
        pencolor="#AEB6BE"
        bgcolor="#E5F5FD"
        subgraph cluster_inventory {
            label="Inventory"
            style=rounded
            style=filled
            pencolor="#AEB6BE"
            bgcolor="#EBF3E7"
            clay_inventory [label="Clay\nInventory"]
            tool_inventory [label="Tool\nInventory"]
        }
        subgraph cluster_staff {
            label="Staff"
            style=rounded
            style=filled
            pencolor="#AEB6BE"
            bgcolor="#EBF3E7"
            clerk [label="Clerk"]
            general_manager [label="General\nManager"]
            general_manager -> clerk [label="supports"]
        }
    }

    // Edges
    mine -> clay_supplier [label="Provides"]
    clay_supplier -> clay_inventory [label="Restocks"]
    tool_supplier -> tool_inventory [label="Restocks"]

    clerk -> clay_inventory [label="Manages"]
    clerk -> tool_inventory [label="Manages"]

    general_manager -> clay_supplier [label="Coordinates\nwith"]
    general_manager -> tool_supplier [label="Coordinates\nwith"]
}

G Sculpting Supply Store cluster_supply_chain Supply Chain cluster_raw_materials Raw Materials cluster_clay_supply Clay cluster_tool_supply Tools cluster_store Store cluster_inventory Inventory cluster_staff Staff mine Mine clay_supplier Clay Supplier mine->clay_supplier Provides tool_manufacturer Tool Manufacturer mine->tool_manufacturer Provides clay_inventory Clay Inventory clay_supplier->clay_inventory Restocks tool_supplier Tool Supplier tool_manufacturer->tool_supplier Provides tool_inventory Tool Inventory tool_supplier->tool_inventory Restocks clerk Clerk clerk->clay_inventory Manages clerk->tool_inventory Manages general_manager General Manager general_manager->clay_supplier Coordinates with general_manager->tool_supplier Coordinates with general_manager->clerk supports

  • Inappropriate orientation and aspect ratio
  • Conspicuous when they are wrong

Challenges Presented by Styling

  • No built-in variable support
  • Analogous to CSS

CSS

body {
  font: 100% Helvetica, sans-serif;
  color: #333;
}

SCSS

$font-stack: Helvetica, sans-serif;
$primary-color: #333;

body {
  font: 100% $font-stack;
  color: $primary-color;
}

Source: SCSS Documentation

React Native

import React from 'react';
import {StyleSheet, Text} from 'react-native';
import {
  SafeAreaView,
  SafeAreaProvider
} from 'react-native-safe-area-context';

const App = () => (
  <SafeAreaProvider>
    <SafeAreaView style={styles.container}>
      <Text style={styles.title}>React Native</Text>
    </SafeAreaView>
  </SafeAreaProvider>
);

const styles = StyleSheet.create({
  container: {
    flex: 1,
    padding: 24,
    backgroundColor: '#eaeaea',
  },
  title: {
    marginTop: 16,
    paddingVertical: 8,
    borderWidth: 4,
    borderColor: '#20232a',
    borderRadius: 6,
    backgroundColor: '#61dafb',
    color: '#20232a',
    textAlign: 'center',
    fontSize: 30,
    fontWeight: 'bold',
  },
});

export default App;

Source: React Native Docs - Stylesheet

Lightweight Solutions

  • IDE Snippets
  • Preprocessors

IDE Snippets

VSCode Open Configure Snippets

VSCode Open Snippets File

VSCode Snippet Graph Template

VSCode Snippet Cluster Colors

Preprocessors

  • Write a script in a lightweight language like
    • Bash
    • Perl
    • Python
  • Script would do text parsing and replacement

More Robust Solution - Wrap in a Language

Uses and Integration with Tools

  • Read from DB to Dynamically Generate Graphs
  • Integrate into documentation
  • Real-time collaboration
  • Other programmatic manipulation

Add to

  • READMEs
  • Plain-Text Knowledge Management Systems

Codebase Inspection

Go-Callvis

Crabviz

Go Callvis - Codebase Inspection

go-callvis -file ecom -graphviz cmd/main.go

gocallvis cluster_focus main cluster_database/sql database/sql cluster_fmt fmt cluster_github.com/sikozonpc/ecom/cmd/api api cluster_github.com/sikozonpc/ecom/db db cluster_log log command-line-arguments.initStorage initStorage (*database/sql.DB).Ping (*DB).Ping command-line-arguments.initStorage->(*database/sql.DB).Ping log.Fatal Fatal command-line-arguments.initStorage->log.Fatal log.Println Println command-line-arguments.initStorage->log.Println command-line-arguments.main main command-line-arguments.main->command-line-arguments.initStorage fmt.Sprintf Sprintf command-line-arguments.main->fmt.Sprintf github.com/sikozonpc/ecom/cmd/api.NewAPIServer NewAPIServer command-line-arguments.main->github.com/sikozonpc/ecom/cmd/api.NewAPIServer (*github.com/sikozonpc/ecom/cmd/api.APIServer).Run (*APIServer).Run command-line-arguments.main->(*github.com/sikozonpc/ecom/cmd/api.APIServer).Run github.com/sikozonpc/ecom/db.NewMySQLStorage NewMySQLStorage command-line-arguments.main->github.com/sikozonpc/ecom/db.NewMySQLStorage command-line-arguments.main->log.Fatal

Crabviz - Codebase Inspection

Crabviz - Codebase Inspection

Obsidian

  • Knowledge management system built-around plain-text
  • Uses text processing of markdown files
  • obsidian.md

Obsidian Inline Graphviz

Obsidian Community Plugins - Graphviz

Obsidian Graph View

Obsidian Graph View - Filtered to Tag

Other Visualizations

Enter, Mermaid

Hello, Mermaid

  • Other visualization tasks

  • Frequent native support in Markdown tools

  • Different rendering style, implications for complicated graphs

  • mermaid.js.org

Example: Sequence Diagrams

sequenceDiagram
autonumber
Alice->>John: Hello John, how are you?
loop HealthCheck
    John->>John: Fight against hypochondria
end
Note right of John: Rational thoughts!
John-->>Alice: Great!
John->>Bob: How about you?
Bob-->>John: Jolly good!

Source: mermaid.js.org

Example: Sequence Diagrams

sequenceDiagram
autonumber
Alice->>John: Hello John, how are you?
loop HealthCheck
    John->>John: Fight against hypochondria
end
Note right of John: Rational thoughts!
John-->>Alice: Great!
John->>Bob: How about you?
Bob-->>John: Jolly good!

Obsidian Inline Mermaid

Example: Entity Relationship Diagram

erDiagram
    direction LR
    CUSTOMER ||--o{ ORDER : places
    CUSTOMER {
        string name
        string custNumber
        string sector
    }
    ORDER ||--|{ LINE-ITEM : contains
    ORDER {
        int orderNumber
        string deliveryAddress
    }
    LINE-ITEM {
        string productCode
        int quantity
        float pricePerUnit
    }

Example: Entity Relationship Diagram

erDiagram
    direction LR
    CUSTOMER ||--o{ ORDER : places
    CUSTOMER {
        string name
        string custNumber
        string sector
    }
    ORDER ||--|{ LINE-ITEM : contains
    ORDER {
        int orderNumber
        string deliveryAddress
    }
    LINE-ITEM {
        string productCode
        int quantity
        float pricePerUnit
    }