Dynamically Resizing Kibana iFrames with Javascript and HTML

19 Jan 2018

If you’re struggling to dynamically resize your Kibana iFrames, you will need two things: Javascript and HTML.

Below is some inline Javascript that will be run onload for each iframe. It is important to note that this script will only work on Kibana tables - not other types of visualizations.

Basically, the script checks to see if the Kibana table has loaded yet - if not, it resets the function with a recursive loop.

Once the Kibana iframe’s table has loaded, we grab that height, and resize the iframe accordingly.

<script>
/* Waits 1 second then checks the iframe again */
function wait_for_load(id) {
  return setTimeout(() => setIframeHeight(id), 1000)
}

/* 
   Called by each iframe onload. 
   Checks if the Kibana table has loaded
   And resizes the iframe when it does
*/
function setIframeHeight(id) {
  if (!id) return;
  const ifrm = document.getElementById(id); // Get our iFrame element using the id
  if (!ifrm) return;

  // If the Kibana iframe table hasn't fully loaded yet, restart this function after a wait
  if (!ifrm.contentDocument || !ifrm.contentDocument.body || 
      ifrm.contentDocument.body.getElementsByTagName('table').length === 0) {
    return wait_for_load(id)
  }

  // Finally! The table has loaded. Let's get the table height and resize the iframe
  const table_height = ifrm.contentDocument.body.getElementsByTagName('table')[0].offsetHeight + 46;
  // The extra 46px of padding helps us avoid IE and Chrome scrollbars on Windows
  ifrm.style.height = table_height + 'px';  
  console.log('We exited the loop for ' + id + ' after setting a height of ' + table_height + 'px');
}
</script>

<!-- Note that the iframes have their onload attribute set to call the above functions -->
<div>
    <iframe id="table1" style="width: 100%;" 
            src="http://your-kibana-source1" 
            frameborder="0" scrolling="no" 
            onload="setIframeHeight(this.id)">
    </iframe>
    <iframe id="table2" style="width: 100%;" 
            src="your-kibana-source2" 
            frameborder="0" scrolling="no" 
            onload="setIframeHeight(this.id)">
    </iframe>
</div>

Improved Jekyll Build Script... Using Bash

18 Jan 2018

I’ve written about my Jekyll build script before. I use this script to deploy this site among others. It is terribly overengineered and unnecessary.

Here is the new and improved version. Instructions for use are below the code itself.

Features
  • JS Minification
  • HTML Minification
  • CSS Minification
  • Thumbnail Generation
  • Intelligent Image Compression
  • Favicon Generation
  • Jekyll Build Process
  • Automatic Upload to S3
  • Setup Mode (Installs Dependencies)
  • Sane Image Names (jpeg, JPG, etc to jpg)
#!/bin/bash
# This script does some various utility tasks
# Builds the static site using Jekyll
# And syncs the generated site with S3

# You can run this script with three options
# -i  | enable Image processing. Creates thumbnails and quickly compresses images.
# -c  | enable maximum Compression for images. Creates thumbnails, thoroughly compresses images, and takes a long time doing it
# -n  | No-upload mode. Doesn't upload the build to S3.
# -s  | enable Setup mode. Downloads the necessary npm files for compression

# SITE OPTIONS - EDIT THESE
SITE_S3='s3://daviseford-website-code/blog/'    # Your S3 bucket address
CSS_DIR='./public/css/'     # Constants
IMG_DIR='./public/img/'     # Constants
JS_DIR='./public/js/'       # Constants
SITE_OUTPUT_DIR='./_site/'  # Constants

# BUILD OPTIONS - EDIT THESE
MINIFY_CSS=true             # Minify any CSS in your CSS_DIR
MINIFY_JS=true              # Minify any JS files in your JS_DIR
MINIFY_HTML=true            # Minify the Jekyll-generated HTML in your SITE_DIR
COMPRESS_IMG=true           # If true, will compress all png and jpg files in the IMG_DIR
RENAME_IMG=true             # If true, will rename files in IMG_DIR from ".JPG" and ".jpeg" to ".jpg"
THUMBNAILS=true             # If true, will create a /thumbnails/ directory in your IMG_DIR
                            # with all of your current IMG_DIR structure copied over

FAVICONS=false              # If true, will generate favicon files for you
                            # Looks at /favicon.png and favicon_cfg.json
                            # Uses https://realfavicongenerator.net/ CLI tool

# END EDITING. DO NOT EDIT PAST THIS POINT.

# CLI OPTIONS - WILL BE SET AUTOMATICALLY. DO NOT TOUCH
ARG_I=false
ARG_C=false
ARG_N=false
ARG_S=false
COMPRESSION_LEVEL="-o1"
BUILD_LOG="build_log.txt"
IMG_THUMB_DIR="${IMG_DIR}thumbnails/"
TMP_THUMB_DIR='/tmp/thumbnails/'
TMP_THUMB_DIR2='/tmp/thumbnails_tmp/'
JPG_OPTS='-resize 445 -sampling-factor 4:2:0' # Will be used by mogrify
PNG_OPTS='445'  # Will be used by imagemagick's convert
# https://stackoverflow.com/questions/3953645/ternary-operator-in-bash
PREVIOUS_BUILD_TIMESTAMP=$([ -f "$BUILD_LOG" ] && echo `stat -f"%Sm" -t "%F %T" "$BUILD_LOG"` || echo "1989-05-22 23:59:59")

# Setting options using getopts
while getopts :icns: opt; do
  case $opt in
    i)  # Image processing
      ARG_I=true
    ;;
    c)  # Compression
      ARG_I=true # Implicit invocation
      ARG_C=true
      COMPRESSION_LEVEL="-o7"
    ;;
    n)  # No upload
      ARG_N=true
    ;;
    s)  # Setup
      ARG_S=true
    ;;
    \?) # Error
      echo "Bad parameter: -i, -c, -n, -s are accepted"
      exit 1
    ;;
  esac
done


# BEGINNING OF BULK OF THE PROGRAM
write_build_log()
{
    current_timestamp=`date '+%Y-%m-%d %H:%M:%S'`
    echo "Built $current_timestamp" > "$BUILD_LOG"
    echo "Created $BUILD_LOG"
}

rename_extension() # This renames files in our IMG_DIR
{
for file in `find ${IMG_DIR} -name "*.$1" -type f`; do
    mv "$file" "${file/.$1/.$2}"
done
}

run_image_tasks()
{
if [ "$RENAME_IMG" = true ] && [ -d "$IMG_DIR" ] ; then
    rename_extension JPG jpg     # Renaming JPG -> jpg (makes the below optimization faster)
    rename_extension JPEG jpg    # Renaming JPEG -> jpg
    rename_extension jpeg jpg    # jpeg -> jpg
    rename_extension PNG png     # PNG  -> png
fi

if [ "$COMPRESS_IMG" = true ] && [ -d "$IMG_DIR" ] ; then # Compress images
    # Only compress if there are new files
    N_JPG=`find ${IMG_DIR} -not -path '*thumbnails/*' -type f -iname '*.jpg' -newerct "$PREVIOUS_BUILD_TIMESTAMP" | wc -l | xargs` # Number of files that meet this criteria
    N_PNG=`find ${IMG_DIR} -not -path '*thumbnails/*' -type f -iname '*.png' -newerct "$PREVIOUS_BUILD_TIMESTAMP" | wc -l | xargs`
    if [ "$N_JPG" -gt 0 ]; then
        echo "Now compressing $N_JPG jpg files in ${IMG_DIR}"
        find ${IMG_DIR} -not -path '*thumbnails/*' -type f -iname '*.jpg' -newerct "$PREVIOUS_BUILD_TIMESTAMP" -exec jpegoptim --strip-com --quiet --max=85 {} \;
    fi
    if [ "$N_PNG" -gt 0 ]; then
        echo "Now running ${COMPRESSION_LEVEL} level compression on $N_PNG .png files in ${IMG_DIR}"
        find ${IMG_DIR} -not -path '*thumbnails/*' -type f -iname '*.png' -newermt "$PREVIOUS_BUILD_TIMESTAMP" -print0 | xargs -0 optipng ${COMPRESSION_LEVEL} -silent # Takes so long
    fi
fi

if [[ "$ARG_I" = true ]] && [ -d "$IMG_DIR" ]; then
    echo "Finished image tasks"
fi
}

minify_html()
{
if [ "$MINIFY_HTML" = true ]  && [ -d "$SITE_OUTPUT_DIR" ]; then
    # Using html-minifier | npm install html-minifier-cli -g
    for file in `find ${SITE_OUTPUT_DIR} -name "*.html" -type f`; do
        htmlmin -o "${file}.min" "$file"  # Make a minified copy of each .html file
        mv "${file}.min" "$file"          # Overwrite the old HTML with the minified version
    done
    echo "Minified HTML"
fi
}

minify_css()
{
if [ "$MINIFY_CSS" = true ]  && [ -d "$CSS_DIR" ]; then
    # Using UglifyCSS | npm install uglifycss -g
    find ${CSS_DIR} -name "*.min.css" -type f|xargs rm -f   # Delete existing minified files
    for file in `find ${CSS_DIR} -name "*.css" -type f`; do
        uglifycss --ugly-comments --output "${file/.css/.min.css}" "$file" # Create minified CSS file
    done
    echo "Minified CSS"
fi
}

minify_js()
{
if [ "$MINIFY_JS" = true ] && [ -d "$JS_DIR" ]; then
    # Using google-closure-compiler-js | npm install google-closure-compiler-js -g
    find ${JS_DIR} -name "*.min.js" -type f|xargs rm -f   # Delete existing minified files
    for file in `find ${JS_DIR} -name "*.js" -type f`; do
        google-closure-compiler-js "$file" > "${file/.js/.min.js}"
    done
    echo "Minified JS"
fi
}

create_favicons()
{
if [ "$FAVICONS" = true ]; then
    if [ -f "favicon.png" ] && [ -f "favicon_cfg.json" ]; then # Make sure we have all our files
        # Using real-favicon | npm install cli-real-favicon -g
        real-favicon generate favicon_cfg.json f_report.json ${SITE_OUTPUT_DIR}
        rm -f f_report.json
    else
        echo "Missing either favicon.png or favicon_cfg.json in the root directory of this site, can't generate thumbnails"
    fi
fi
}

create_thumbnails()
{
if [ "$THUMBNAILS" = true ] && [ -d "$IMG_DIR" ] ; then
    rm -rf ${TMP_THUMB_DIR} && mkdir ${TMP_THUMB_DIR}  # Housekeeping
    rm -rf ${TMP_THUMB_DIR2} && mkdir ${TMP_THUMB_DIR2}  # Housekeeping
    rsync -a --exclude '*thumbnails/*' --exclude '.DS_Store' ${IMG_DIR} ${TMP_THUMB_DIR}  # Move images to /tmp/
    find ${TMP_THUMB_DIR} -name '*.DS_Store' -type f -delete # Delete pesky .DS_Store files
    EXISTING_FILE_COUNT=`find ${TMP_THUMB_DIR} -type f | wc -l | xargs`

    # A.) Check that the file has an associated thumbnail file in our IMG_THUMBDIR
    # B.) If it does, just copy that file to our TMP_DIR2
    # C.) If it doesn't, resize and compress
    find ${TMP_THUMB_DIR} -type f -not -newermt "$PREVIOUS_BUILD_TIMESTAMP" | while read file; do
        FILEPATH="${file/${TMP_THUMB_DIR}\//}"
        DIRNAME=$( dirname "$FILEPATH" )
        DEST="$TMP_THUMB_DIR2$FILEPATH"
        MKDIR="$TMP_THUMB_DIR2$DIRNAME"
        ORIG_THUMB_PATH="$IMG_THUMB_DIR$FILEPATH"
        if [ -f "$ORIG_THUMB_PATH" ]; then # File exists already, just copy it to $DEST, and delete the file in our TMP_THUMB_DIR
            mkdir -p "$MKDIR" && mv "$ORIG_THUMB_PATH" "$DEST" && rm -f "$file"
        fi
    done

    NEW_FILE_COUNT=`find ${TMP_THUMB_DIR} -type f | wc -l | xargs`

    # Resize thumbs
    find ${TMP_THUMB_DIR} -type f -iname '*.jpg' -exec mogrify $JPG_OPTS {} \;
    find ${TMP_THUMB_DIR} -type f -iname '*.png' -exec convert \{} -resize $PNG_OPTS\> \{} \;

    # Further compress thumbnails
    find ${TMP_THUMB_DIR} -type f -iname '*.jpg' -exec jpegoptim --strip-com --quiet --max=85 {} \;
    find ${TMP_THUMB_DIR} -type f -iname '*.png' -print0 | xargs -0 optipng -o7 -silent # Takes so long

    # Move the old thumbnails (already compressed) back into our dir
    rsync -r --size-only ${TMP_THUMB_DIR2} ${TMP_THUMB_DIR}

    # Move TMP_THUMB_DIR thumbnails into the IMG_THUMB_DIR directory - only overwrite if size is different
    rsync -r --size-only --delete ${TMP_THUMB_DIR} ${IMG_THUMB_DIR}
    rm -rf ${TMP_THUMB_DIR} && rm -rf ${TMP_THUMB_DIR2} # Delete our temporary working directories
    echo "Generated $EXISTING_FILE_COUNT thumbnails ($NEW_FILE_COUNT new) in ${IMG_THUMB_DIR}"
fi
}

### START OF EXECUTION ###
# Run setup
if [ "$ARG_S" = true ]; then
    brew install imagemagick
    npm install google-closure-compiler-js -g
    npm install uglifycss -g
    npm install html-minifier-cli -g
    npm install cli-real-favicon -g
    exit 0
fi

# Run this script with the "-i" flag to process images (takes longer)
if [ "$ARG_I" = true ]; then
    create_thumbnails && run_image_tasks
fi

# Minify CSS and JS source files - important to do this BEFORE building
minify_css && minify_js

# Build with Jekyll
bundle exec jekyll build

# Minify HTML (this modifies the generated HTML) - do AFTER building
minify_html

# Create favicons
create_favicons

# Upload to S3 - unless -n (no-upload) is passed in
if [ "$ARG_N" = false ]; then
    aws s3 sync --delete --size-only ${SITE_OUTPUT_DIR} ${SITE_S3} --exclude "*.sh" --exclude "*build_log.txt"
    echo "Uploaded to S3"
fi

# Write our build log
if [ "$ARG_I" = true ] && [ "$ARG_C" = true ]; then
    write_build_log
fi

echo "Done!"
How to Use

Copy the script into a new text file and save the file in the root of your Jekyll directory as build.sh.

Make sure to set up the options under # BUILD OPTIONS - EDIT THESE in build.sh.

Install dependencies by running sh build.sh -s. Requires homebrew.

Then, run the build script by running sh build.sh -i -c.

Sample Build Output
daviseford$ sh upload_to_s3.sh -i -c
Generated 126 thumbnails (1 new) in ./public/img/thumbnails/
Now running -o7 level compression on 1 .png files in ./public/img/
Finished image tasks
Minified CSS
Configuration file: /Users/davisford/Documents/daviseford-jeykll/_config.yml
            Source: /Users/davisford/Documents/daviseford-jeykll
       Destination: /Users/davisford/Documents/daviseford-jeykll/_site
 Incremental build: disabled. Enable with --incremental
      Generating...
         AutoPages: Disabled/Not configured in site.config.
        Pagination: Complete, processed 1 pagination page(s)
                    done in 3.856 seconds.
 Auto-regeneration: disabled. Use --watch to enable.
Minified HTML
upload: _site/public/img/misc/aws_cert/AWS_Certified_Logo.png to s3://daviseford-website-code/blog/public/img/misc/post_jan_crash.png
upload: _site/public/img/thumbnails/misc/aws_cert/AWS_Certified_Logo.png to s3://daviseford-website-code/blog/public/img/thumbnails/misc/post_jan_crash.png
Uploaded to S3
Created build_log.txt
Done!

Any problems, just email me.


A Klein Bottle from Cliff Stoll

17 Jan 2018

I purchased a Klein Bottle from Cliff Stoll on January 16th, 2018 - a cold, bitter morning. I sent him this note, along with my credit card number and presumably all of the information needed to ruin my life.

Cliff, I hope you read this. I was born in 1989 and grew up reading the Cuckoo's Egg.[1] I even baked the chocolate chip cookies you were so kind to provide the recipe for.[2]

Many years later I am a software developer and professional tinkerer. I was profoundly influenced by your book and your mad hatter approach to computer science.

I am sure you hear this often, but I thank you very much for your contribution to my world.

I am different, and better, for having ingested your peculiar way of conveying stories. Even reading this website had me in stitches.

Happy to have purchased something with sentimental value, I worked for the next few hours.

By lunch, I had received a reply from the man himself.

Hi Davis,

Thank you for your smiling comments about Cuckoo's Egg - oh, but that sends me back a quarter century (when there were payphones, modems, and pocket pagers). You actually tried those chocolate chip cookies? The back story: we used to make these once or twice a week. Well, while writing that chapter, I was making a batch of 'em and got some butter or grease on my keyboard. Figuring that this was some kind of omen, I decided to put it right into the book, at the very place where I was writing. So, in an indirect way, you were just following the storyline of my life in the 1980's. I'm especially happy to hear that you tinker around with software (and, I assume, hardware) ... so much of software work seems to be just library calls and managing releases. Gone are the days of building things...

Thanks, of course, for your Klein bottle order! I've just signed your medium sized Klein bottle and then snuggled it into a 9x5x5 inch box along with an invoice and the usual Acme topological propaganda. I scribbled a greeting on the packing slip as well.

While packing the glass manifold, I took a few photos, several of which seem to be in focus. In my next email, I'll send these photos to you. But if you're on a slow internet connection, there's no need to download them - the files may be fairly large.

I'll bike to the post office today (Tuesday the 16th) and send it via First-Class Mail. In theory, the box oughta arrive on Friday January 19, give or take an aeon. It's hard to predict exactly, what with the weather, the political situation, the economy, global warming, continental drift, the expansion of the universe, and the uncertainty principle.

So, from across the continent and around our three spatial dimensions, here's my one-sided cheers to you. I'm sure you'll like the Klein bottle - and my warm wishes that you put together a few stories...

-Cliff (on a misty Tuesday morning in Oakland)

I was stunned to get a personalized touch - and I got to hear about the book that I grew up devouring.

A second email sat in my inbox. Inside were pictures of my Klein bottle teleporting around Cliff’s house. We traveled from his workshop to his wife’s garden.


His workshop, by the way, has been a huge YouTube hit for years. This video will delight you if you’ve never seen it.


The reason I was so enthralled to hear from Cliff? He’s one of the men who taught me to troubleshoot.

The Cuckoo’s Egg is all about tracking down the tiniest details using the software and hardware available at the time. Cliff races baud modems and phone trunks to pinpoint his guy - all because of a seventy-something-cent error in the Berkeley accounting program. The book is a terrific read and extraordinarily detailed. The amount of patient troubleshooting displayed throughout the book was foundational for my understanding of how to take apart a computer-driven jigsaw puzzle.

I have never read such a thoroughly entertaining book. Cliff’s mad scramble to deal with printers and track keystrokes and save emails - hysterical. I’ve often thought of Cliff while debugging RealSimpleEmail, my email platform. There were a lot of strange routing bugs that I needed to learn about the email world, and I felt kinship with a young Cliff Stoll, poking around in areas he probably shouldn’t in order to understand the bigger picture.

Cliff Stoll was a childhood hero of mine because of Cuckoo’s Egg.

He was an adolescent hero of mine because I began my career tinkering with computers and understanding the complexity of the systems we have today.

He is an adult hero of mine because he took time to respond to a lifelong fan.

I thought it would be only fitting, since he was sharing his art with me, that I share some with him.

Given that I’ve been tinkering with generating word art from novels, I figured it was only natural to run Cuckoo’s Egg through my script.[3]

I emailed him this little piece of generated art. I do hope he enjoys it.

The Cuckoo's Egg - 1,349 sentences
Sentences containing 'hacker' are highlighted red.
Start/End = Light/Dark Green Dot

  1. The Cuckoo’s Egg: Tracking a Spy Through the Maze of Computer Espionage by Cliff Stoll
  2. “Two eggs, 1 cup brown sugar, 1/2 cup regular sugar, 2 sticks softened butter. Fold in 2 1/4 cups flour, 1/2 teaspoon salt, 1 teaspoon baking soda, and a couple tablespoons of vanilla. For an extra chocolate jag, toss in 3 tablespoons of cocoa. Oh, don’t forget 2 cups of chocolate chips. Bake ‘em at 375 degrees for 10 minutes.”
  3. The script calculates the number of sentences in a given text input (1,349 in this case). Once the script has reduced the book to a list of sentences, we then transform those sentences into a number representing the number of words in a sentence. The previous sentence, for example, would be represented by 28 (28 words). We then draw a path using Python and SVG. Each straight line represents a sentence in the book - and its length is determined by how many words are in the sentence. At the end of each sentence, the path turns left 90 degrees.

SCSS Media Queries - Responsive Breakpoint Mixins

10 Jan 2018

Here are some utility mixins for SCSS that you can use for quickly mocking up responsive CSS.

// Screen size variables
$screen-xs-min: 425px;  // Tiny phones
$screen-sm-min: 576px;  // Small tablets and large smartphones (landscape view)
$screen-md-min: 768px;  // Small tablets (portrait view)
$screen-lg-min: 992px;  // Tablets and small desktops
$screen-xl-min: 1200px; // Large tablets and desktops

// Mixins
@mixin xs { @media (min-width: #{$screen-xs-min}) {@content;} } // Tiny devices
@mixin sm { @media (min-width: #{$screen-sm-min}) {@content;} } // Small devices
@mixin md { @media (min-width: #{$screen-md-min}) {@content;} } // Medium devices
@mixin lg { @media (min-width: #{$screen-lg-min}) {@content;} } // Large devices
@mixin xl { @media (min-width: #{$screen-xl-min}) {@content;} } // Extra large devices

Use like so:

/* A class that will respond differently in different resolutions */
.sample-padding-class {
  
  padding-bottom: 200%; // Default property, this will be applied to all screen sizes
  color: #FFF;          // Unless overriden below
  
  @include sm {         // 576px window width and more
    padding-bottom: 100%;
    color: #000;
  }
  
  @include md {         // 768px window width and more
    padding-bottom: 50%;
  }
  
  @include lg {         // 992px window width and more
    padding-bottom: 10%;
  }

  @include xl {         // 1200px window width and more
    padding-top: 20%;
    padding-bottom: 0;
    color: #1337;
  }
}