Notes about drying filament

 

CNC kitchen on drying with desiccant- https://www.cnckitchen.com/blog/cyo43tzz88uqge65xgwz0wv8yvv3rs

https://www.reddit.com/r/3Dprinting/s/q4ynPjmASl 

This is everyone's monthly reminder that "Hygroscopic" and "Hydrophilic" are two different classes of material.

Hygroscopicity is a mechanical process and does not involve bonds that need to be broken. PLA filaments are hygroscopic, as are most thermoplastics. Think of it like a sponge soaking up water - it's just entering voids in the material.

Hydrophilicity is a chemical process that involves bonds that need to be broken. Nylon is hydrophilic, as is polycarbonate (slightly).

Both are important, as non-hydrophilic plastics like PETG and PLA can have strongly hygroscopic structures introduced by additives and the process of extrusion.

Heat is not required to reduce the water content of hygroscopic plastics, but it accelerates the process by increasing the vapor pressure of trapped water.

Heat is required to reduce the water content of hydrophilic plastic filaments. Bonds have to be broken.

No condescension is intended, it's tangled in a lot of jargon and is a bit of a nightmare to get clear info on.

( Fan, N. X. (2008). Mechanics of moisture for polymers: Fundamental concepts and model study. 9th. Int. Conf. - IEEE, 22, 1–14. https://doi.org/10.1109/esime.2008.4525043

Sperling, L. H. (2005). Introduction to Physical Polymer Science. John Wiley & Sons. )

https://www.reddit.com/r/3Dprinting/s/1ejli6Ju34   Hydrolysis causes irreversible polymer damage. You can dry it out a few times and not notice much difference. Performance degrades with every hydration and drying cycle, some H20 bonds on polymer surfaces, and dried filament has higher surface area. After around 5 years PLA stored in atmospheric humidity will become too brittle to print with, no matter how many times it's been dehydrated. Keep dry filament dry, or dry it then print it fast.

https://www.reddit.com/r/3Dprinting/s/8DG0zSL6Sf

Hi, I am a polymer chemist, and can shed a bit of light here. None of this is to say that the state of the filament isn't a contributor, or even the main contributor, to the problem, but there are legitimate chemical changes too. Water causes hydrolysis in the PLA backbone, which reduces the molecular weight of the polymer. Generally, as molecular weight of a polymer increases, the mechanical properties increase with it, so if it is reduced, so are the mechanical properties. There are two major ways this can happen; slowly over time, as PLA is fairly polar and interacts with water, so it will take some up over time, and the second would be when you are actually printing (processing) the polymer. The increased temperature makes whatever water is there able to hydrolyze much more quickly, so even if the filament isn't actually hot long, it takes much less time to degrade the molecular weight. I honestly have no clue what proportion of the problem each of those "pathways" has, so am not sure which would be a more practical problem, the storage, or making sure it is dry before processing.

I do have an off the cuff explanation for why drying could make it more brittle too, though. While water can cause the covalent bond breakage in the PLA polymer, it can also act as a plasticizer(which is often a desirable thing, but not in this case since the "plasticizer" is also able to degrade the polymer). This means that, only when the water is still actually present, intermolecular interactions between the pla and water can make the plastic more flexible (decreasing the modulus). When OP removed the water, I'm guessing it shined a light on the degradation that had already happened. I don't think the OP made a mistake by drying their filament. If they run it through their heated nozzle, my guess is the water would just compound the issue at the extruder head, they just wouldn't know it until they either had issues printing or had a brittle product.

If you have any questions or need any clarifications let me know, I'm happy to explain as best I can!

Commentary about drying filament - https://www.reddit.com/r/3Dprinting/s/bABDM0dEj6

https://www.reddit.com/r/3Dprinting/comments/1n7va3n/heres_some_tests_showing_how_well_sillica_gel/

https://www.reddit.com/r/3Dprinting/comments/1jgpwdq/yes_you_really_can_dry_filament_with_silica_gel/

Minimizing/removing wipetower

 Putting this here for reference - a thorough example of making wipe objects and not needing a purge tower in Bambu studio.

https://forum.bambulab.com/t/making-better-use-of-flush-into-object-in-bambu-studio/29422

Referenced from https://www.reddit.com/r/3Dprinting/s/XGbFlV9CtS

Additional reference for multiple material wipe towers : https://forum.prusa3d.com/forum/original-prusa-xl-tool-changer-general-discussion-announcements-and-releases/separate-wipe-tower-for-different-material-or-print-perimeter-first-on-wipe-tower/ 

Wipetower tempature fix when using multiple materials with the MMU

 Due to a bug (with a longstanding, updated PR) on the temp change happening after a filament swap instead of before it causing clogs when higher temp filament hits a cold nozzle, coupled with the difficulty building PrusaSlicer yourself, they've started pushing up their own builds/releases on their fork of Prusaslicer - lets hope that this PR gets merged into core soon so this workaround isn't required!

https://github.com/amatulic/PrusaSlicer/releases/tag/wipetower

MMU3 Gotcha with Octoprint!

 When setting up the MMU3, I forgot to (didn't realize that I need to) configure octoprint to see that my printer had more tools, so it was only printing in one color (and asking me to select it) instead of switching between the tools. - https://community.octoprint.org/t/error-t-command-was-suppressed-and-not-sent-to-printer/27155

Reverse Bowden

 Really good description of what a 'reverse bowden' setup is for a direct drive printer. - https://forum.prusa3d.com/forum/english-forum-general-discussion-announcements-and-releases/reverse-bowden/ 

To understand what a reverse bowden is, first you need to understand what a regular bowden and direct drive are.

A bowden is a setup where the extruder drive motor is mounted in a fixed place on the frame of the printer. The flexible tube connecting the extruder drive motor to the hot-end and nozzle is referred to as a bowden tube. This tube constrains the length between the drive and the nozzle so when the drive pushes on the filament the force is applied to the hotend. This enforces that when length x is pushed through the drive, the same length x is pushed into the melt zone to have a controlled amount of melted filament pushed out of the nozzle. In this configuration, the only way to pull filament off the spool is by the extruder drive motor pulling on filament because it is presumed that you aren't moving the spool around during a print relative to where the frame of the printer is.

On a direct drive setup the extruder drive motor is mounted to the moving hot-end and nozzle. In these configurations the distance between the extruder drive motor and the hotend is fixed, usually around a couple inches in distance, and almost always in a rigid fixed straight path. (I don't know of any that aren't in a straight path, but I don't know every extruder out there including experimental extruders.) In this configuration, not only does the extruder drive motor pulling on the filament off the spool, but the motion of the extruder assembly can pull filament off the spool if it moves away from where the spool is. This can cause situations where the extruder assembly moves closer to the spool, essentially trying to push the filament back onto the spool. But because the spool won't rewind (unless you have a special rewinder type spool holder) this extra slack can allow a loop of filament to fall off of the spool, potentially causing tangles to happen with the loose filament on your spool holder.

The reverse bowden is one of many ways to mitigate the last issue in the previous paragraph. It is a bowden tube that is connected from the frame of the printer to the input of the extruder drive motor. This constrains the distance between the frame of the printer and the drive motor so the motion of the extruder assembly doesn't change the distance from the extruder assembly and the spool. Often if your direct drive printer doesn't have a reverse bowden then someone out there has designed a setup and you can download the bracket parts that fit on your printer from Thingiverse or similar sites to print them out yourself.

I've played with the idea of installing a reverse bowden on my i3 MK3S, but have yet to have a failure related to not having a reverse bowden, but then again I still use the spool holder that came with the printer so the spool is above the print area. This reduces the amount of potential tangle points along the filament feed path.

BTW, if you do go for the MINI, that is a bowden drive setup so there is no need to add a reverse bowden setup to it.

See my (limited) designs on:
Printables - https://www.printables.com/@Sembazuru
Thingiverse - https://www.thingiverse.com/Sembazuru/designs

Another description of reverse bowden - https://e3d-online.com/blogs/news/battling-bowden-tube-physics 

Search Reddit saved history

 Random useful link here - more and more of my reading lately has been sending things into the blackhole of 'saved' posts on reddit, and they don't even have a search bar! Stumbled on this page that lets you log in, and uses the API from the browser to fetch your history and make it searchable for you!

https://github.com/ahmed-zubair-1998/Saveddit

Misc 3d Printing references.

 Working on keeping track of a few interesting posts that I've found around the internet..

I'll be throwing together some links to random bits of information with respect to 3d printing - Links to reddit posts, printables/thingiverse/thangs links for future tracking, all because my personal notes and reddit histories are not very useful for tracking down information!

General 3d printing guides:

https://teachingtechyt.github.io/calibration.html#firstlayer

How to change hot end - 3D printer model https://m.youtube.com/watch?v=OzRAVkXjw3I

3d design guides

https://www.youtube.com/watch?v=Hna4V7-FJ08

Drybox ideas that aren't on printables/thingiverse

https://3d-print-files.com/instruction-diy-filament-dry-box-the-anybox-v2/

https://www.builditmakeit.com/projects/3d-filament-drybox

Original Lack V2 Blogpost - https://blog.prusa3d.com/mmu2s-printer-enclosure_30215/ 

Dry buffer ideas - https://forum.prusa3d.com/forum/original-prusa-i3-mmu3-user-mods-enclosures-nozzles/mmu3-how-to-keep-filament-slack-dry-in-buffer/

Sunlu s4 ideas:

https://forum.prusa3d.com/forum/original-prusa-xl-tool-changer-user-mods-octoprint-enclosures-nozzles/sunlu-s4-setup/

Use couplers to pass through - https://www.amazon.com/dp/B07MLJYQYR?ref=cm_sw_r_cp_ud_dp_CDPAHAFRHYVVWBFRYAMP&ref_=cm_sw_r_cp_ud_dp_CDPAHAFRHYVVWBFRYAMP&social_share=cm_sw_r_cp_ud_dp_CDPAHAFRHYVVWBFRYAMP

Controlling brim placement with prusaslicer -

https://m.youtube.com/watch?time_continue=12&v=0Zxl1WOJm90&embeds_referring_euri=https%3A%2F%2Fwww.google.com%2F&source_ve_path=Mjg2NjY

 https://m.youtube.com/watch?v=dhsbMmosbLU&feature=youtu.be - https://www.reddit.com/r/prusa3d/s/9FLv8rOtII

https://m.youtube.com/watch?v=0Zxl1WOJm90&t=12s&pp=2AEMkAIB - this one works, under channel slicer underground - https://m.youtube.com/@Swiss_Cheese.

Stop Warping - https://m.youtube.com/watch?v=iPZoDltS30A&pp=0gcJCWoCo7VqN5tDiggUQAFKEGRUOHI3UUF1RU4tRmVuSHY%3D&ra=m

Sequential bridging - https://m.youtube.com/watch?v=KBuWcT8XkhA

Untangle filament - https://www.youtube.com/watch?v=lE9LchCtKL4

Another untangle example - https://www.reddit.com/r/3Dprinting/s/XCrq2cYtQu

More tangled filament discussion - https://www.reddit.com/r/3Dprinting/s/s7ndQly5ho

I need to go against the crowd here. This is not user error. I've had the same problem with eSUN's ePLA+ and PETG.

The issue is not tangled filament.The issue is that the spool was not loaded tidily. There was probably slack in the filament spool during the winding process. Normally, filament that will be extruded first are in the outer layers after winding. The slack during production allowed the filament strands to mix enough such that filament to be extruded later ended up in the outer layers, whereas filament extruded first ended up in the inner layers. During the packaging after the winding process of production, they pulled to tighten the filament around the spool, thus locking in this aberrant arrangement of filament around the spool.

The outcome is when you feed the filament into your extruder, you are pulling on untangled but buried filament. If there is not enough force to unbury the filament, it will fail to feed as in OP's post. For me, the unburying process unwound too much of the spool, leading to too much slack, causing filament getting stuck in the axle of my spool holder.

You can identify these problematic spools because out of the packaging, they are not neatly wound. I know this is not tangled filament, because I can feed an entire problem spool without once unloading it, if I keep fixing the problem by manually "unburying" the filament. And no, and the end of the spool there is not knot or crossing strands of filament.

https://www.reddit.com/r/FixMyPrint/s/GODhRnaQB9

Drying/ brittle filament : https://www.reddit.com/r/3Dprinting/s/8ehTVV0b02

PLA is very prone to hydrolysis - where the adsorbed water molecules literally cut up the long polymers of the plastic. Additives to PLA (silk/fibres/matte/etc) tend to accelerate this chemical breakdown.

If drying doesn’t bring it back to reasonable, it's likely they're hydrolysed to death and ready for the bin.

Connector settings - https://www.reddit.com/r/printablescom/s/bkTvskOE6f

How to orient boxes - https://www.youtube.com/watch?v=8NKVNwVaZU0

Overlapping layers between parts - 

Here is a way to make two imperfect pieces that interlock to not show light between them - commonly used for injection molded design, but can make nice 3D prints https://www.youtube.com/watch?v=8dhFhU7Nl_0

How to split an STL into parts in PrusaSlicer - https://3dprinting.stackexchange.com/questions/21777/prusaslicer-how-do-i-preserve-the-location-of-interlocking-parts-in-a-muliti  (found that after looking for an answer to this - https://www.reddit.com/r/prusa3d/comments/1ec3e6a/comment/lexfa7g/ - https://forum.prusa3d.com/forum/original-prusa-i3-mk3s-mk3-how-do-i-print-this-printing-help/overlapping-objects-in-prusa-slicer/ )

Information around multi-material printing with an MMU

https://www.reddit.com/r/prusa3d/comments/1cm8knv/comment/l2za6gg/ - First, make sure you are in Advanced / Expert mode, these are considered Advanced settings.

Print Settings -> Multiple Extruders

In the "Extruders" section, you can set the extruder being used for various parts of the print and tower. "Wipe tower extruder" controls which one is being used for the outer wall of the tower.

-

You also must set the support distance to “0.0 (soluble)”, otherwise it gives a warning about not supporting multiple extruders with the wipe tower.

Of course this is exactly what you want, but if you don’t set the support gap to zero before assigning extruders, it looks like it doesn’t work.

Issues with layer shift on tall prints - https://www.reddit.com/r/prusa3d/s/T5n42RZRWk

TPU notes: 

To fix stringing: https://www.reddit.com/r/3Dprinting/s/cL6g0mgSZL

“ "Coasting" in PrusaSlicer

Cura has a very useful setting called "coasting" which tells the extruder to stop pushing out filament just before the end of its print move; this allows any ooze to become used in the print instead of becoming blobs and strings during the travel move. PrusaSlicer does not have this setting, but you can achieve a similar result with the following printer settings: set a non-zero retract length, enable "Wipe while retracting", then set the "Retract amount before wipe" to 100%. This will make the printer do all its retraction during the last bit of the print move, which roughly mimics the "coasting" ability of Cura, and it works very well for TPU, shown below. My retract length for this improvised "coast" setting was just 0.4mm. But also, calibrate your linear advance: mine is 0.2 for a 0.4mm brass Revo 6 nozzle. ”

Pause and insert pre-printed support- Tpu https://blog.prusa3d.com/10-tips-for-saving-filament-money-and-environment_71415/

Twisting filament 

https://forum.prusa3d.com/forum/english-forum-original-prusa-i3-mk4-assembly-and-first-prints-troubleshooting/filament-twists-during-print/

Discussion around the S4 - adding connectors to den all 4 out one side - https://forum.prusa3d.com/forum/original-prusa-xl-tool-changer-user-mods-octoprint-enclosures-nozzles/sunlu-s4-setup/

https://forum.prusa3d.com/forum/original-prusa-xl-tool-changer-user-mods-octoprint-enclosures-nozzles/sunlu-s4-setup/?language=it

https://www.reddit.com/r/prusa3d/s/pNxdAG4T8J   Gcode to remove dwell - 

Adding nextruder to mk3.5 - https://www.reddit.com/r/prusa3d/s/KYPZ9C9ZKW

Edge curl / warping example with different bed temps - 

https://www.reddit.com/r/3Dprinting/s/QFSziaLl6L

Prevent warping - https://m.youtube.com/watch?v=iPZoDltS30A&pp=iggCQAE%3D&ra=m

Different warping video https://www.youtube.com/watch?v=wiSyyk2Nv-A

prusa blog about using place in supports for TPU printing - https://blog.prusa3d.com/10-tips-for-saving-filament-money-and-environment_71415/ 

Issue with blob on filament swap - https://forum.prusa3d.com/forum/original-prusa-i3-mk3s-mk3-how-do-i-print-this-printing-help/using-mmu-profile-without-the-mmu-unit-for-manual-multi-material/

Ultimaker profile for prusaslicer - https://ansonliu.com/2024/04/ultimaker-s3-s5-s7-prusaslicer-profile/

Installing gems in PDK on airgapped/offline system without functional rubygems.org

 I ran into a problem recently installing gems in an offline system... I am not understanding something about how the `pdk bundle` setup creates it's cache directories.

If you try to do a `pdk bundle exec gem install ~/path/to/module.gem`, it will place it's cache files in ~/.gem/ruby, but the PDK expects it's files in ~/.pdk/cache/ruby/

I followed these basic steps to get it 'working' enough...

1. Copy over .gem file and it's dependencies to the offline system to be able to do a local `gem install`
2. Install it using `pdk bundle gem install ~/gems/*`
3. Copy over the installed gems from ~/.gem to ~/.pdk/cache - `cp --no-clobber ~/.gem/ruby ~/.pdk/cache/`
   
4. Update your Gemfile (in your PDK base directory) to include the gems you want to local install
5. `pdk bundle install --local` to install the gems from the gemfile, but source them from the cache that you just manually populated.  

I'd love to know if there is a better way to do this!

Compare a whole file content with rspec-puppet

 I was attempting to use the entire body of a file for a quick and dirty 'golden master' style test while I was refactoring some things.

I ran into a problem with having a multi-line string representing the full body of my output - I would run into this problem after putting in something like the following( https://github.com/puppetlabs/rspec-puppet/issues/100 ):

Failure/Error: it { is expected.to contain_file('/mock/file.sh').with_content(multiline_content) }
  expected that the catalogue would contain File[/mock/file.sh] with content set to supplied string
  Diff: 
    <The diff is empty, are your objects producing identical '#inspect' output?>

And my test looked something like this ...

my_content=<<-CONTENT.chomp
file line one
file line two
file last line
CONTENT

on_supported_os.each do |os, os_facts|
  context "on #{os}" do
  it { is_expected.to contain_file('/mock/file.sh').with_content(my_content) }
  end
end

Since I was unable to figure out a 'proper' solution (with and without the chomp) for this problem, I did the next best thing - I simply looped over each line of the file input, and made sure all the lines were in the content - this could give a false 'passing' as it doesn't prevent extra lines from being in there, or the line order isn't tracked, but it was 'good enough' for my short purpose

my_content=<<-CONTENT.chomp
file line one
file line two
file last line
CONTENT

on_supported_os.each do |os, os_facts|
  context "on #{os}" do
  it {
    my_content.each_line do |line|
      is_expected.to contain_file('/mock/file.sh').with_content(/#{Regexp.quote(line)}/) 
  }
  end
end

Hope this helps someone, or perhaps someone see's this and can tell me what I'm doing wrong!

Into the world of 3d printing

I have long been interested in 3d printing, but I have mostly had other people print for me.  I got myself a Prusa MK3s+, and have been doing the SD card shuffle to get my prints to the printer, which has been working flawlessly.  

I have a raspberry pi lying around, but for reasons I cannot figure out, my previous install had docker, docker-compose, and k3s installed on it.  Whenever I accessed the pi, I would always get a 404, and I didn't see any processes listening to 80/443 on my system.  There was probably something that the k3s install had done, but I wasn't able to track it down.  

To verify that I didn't have some other phantom issue on my network, I pulled down the latest copy of raspbian, cloned https://github.com/OctoPrint/octoprint-docker and went to install docker-compose, using the way I knew (which was apparently the 'old way'), through pip.

I tried to `sudo pip3 install docker-compose` and ran into an error that I hadn't seen before when installing python modules.... `error: can't find Rust compiler`...  One of the dependencies of docker-compose is the cryptography package, and the newest version that was being pulled in now requires rust to build it.  The error message itself and various resources online said they've fixed it by updating pip... `sudo pip3 install --upgrade pip`, but that didn't solve my problem.  

I installed the rust tools `sudo pip3 install setuptools_rust` but it still failed the same way.  I installed the packages for my raspberry pi using apt( `sudo apt-get install cargo`, but they had to old of a compiler... The dependency required 1.56, and the package installed for raspbian was 1.42.

I followed the instructions from the error installing the crypto module, and that wanted me to use `rustup` to manage my rust version.  I pulled down the rustup installer, and that went fine, but when I tried to install docker-compose with sudo (a mistake, but oh well, I'll be reinstalling the OS onto my harddrive instead of an SD card later), the install of docker-compose still failed, due to my user having rustc on their path, but the root user not having it.  So I switched to a root shell, installed rustup, and was able to run my pip install of docker-compose.

As a side path, while I was trying to figure out how to install docker-compose, I saw some mention that the python version was v1, and v2 was a statically linked go binary to run as a docker plugin.... which led me to having to do a little  bit of manual fighting with https://github.com/docker/compose ... The install script was detecting me as arm64, but I had installed the 32 bit OS, and my system reported as arm7.  So I had to manually download the correct build, and update the binary to use that instead of what it pulled down.

the Octoprint-docker guides are all using the old docker-compose syntax, so I needed to use  utility called compose-switch to convert my syntax for me..... one problem, they didn't build the arm7 binary, only amd64 and arm64 (I had installed the 32 bit os..) 

So no problem right?  Someone even had a snippet in a github issue to cross-compile it using the go toolchain in a docker run on my laptop then copy it over to my pi...   But when I was following the advice in the last comment in the issue thread, they mentioned that the initial poster had the path wrong, but I was getting an error during the `docker cp` step... After a bit of digging, it turned out the initial suggestion of copying from /go/bin/linux_arm/compose-switch was correct, and /go/bin/compose-switch was wrong.  Once I fixed that up, I could copy over the compose-swich binary and manually follow the setup steps to set compose-switch as the etc-alternatives for docker-compose, which would remap my commands to `docker compose`.  

Once I fixe all the above, I was able to run `docker-compose up -d` and see octoprint running on my pi!

I didn't have my printer plugged in yet, but I was seeing the interface and not getting a 404 when trying to contact my pi.  

I did want to verify that the webcam worked properly, and I had to remember to go into `raspi-config` and enable 3. Interface Options and I1 Legacy Camera Enable.  Once I did that (and rebooted), and updated my paths for my camera according to the octoprint docker image readme, and added the right env vars and devices to my docker compose file (uncommented from the octoprint-docker provided docker-compose.yml file, I was able to see my camera stills/stream from within the Octoprint UI.

Lots of jumping around for something that I thought was going to be about 3 commands from a fresh raspbian install!

Shaking off the dust

 I've resolved to start keeping track of helpful information again, so reconfigured access to the ol' blog.  

Working on copying down some videos to an old phone for our daughter to watch in airplane mode so the main phones aren't used/abused/reconfigured...   There are a few songs that are more bearable at different playback speeds.. Stumbled upon this comment on a thread that gave a good example of modifying both the audio and video playback speeds at the same time with a complex filter in ffmpeg.

https://davidwalsh.name/video-speed#comment-509672

Relevant code snippet, to use the same 'multiplier' even though in one case it's a divisor.  

ffmpeg.exe -i input.mp4 -filter_complex "[0:v]setpts=PTS/1.3[v];[0:a]atempo=1.3[a]" -map "[v]" -map "[a]" output.mp4

Today, I was trying to experiment a bit with the go programming language.  I wanted to get started, but the 'install' section of the golang site seemed a bit daunting, and I figured I should use a tool similar to 'RVM' or 'rbenv' from the ruby space for go.  I discovered https://github.com/moovweb/gvm/, but ran into a small hiccup.  The install instructions were to pipe some commands into a bash shell, but after running them, the `gvm` command appeared to be invoking something else!  

From the output, it appeared to be sdkman, which previously was called gvm for 'groovy version manager'. It appears that the install of sdk man places it's files into ~/.sdkman, in addition to calling thing in all three of ~/.profile, ~/.bashrc, and ~/.bash_profile.  I had to explicitly clear out the statements sourcing the sdkman binaries to allow gvm to function properly.  

After clearing out sdkman, I was able to successfully get the latest go installed (after installing go 1.4 from a binary, due to the source compile failing).  `which gvm` now pointted at the go gvm, instead of sdkman!

rspec-puppet test with array parameter

Today I stumbled onto something that was much harder than I expected it to be while trying to write an rspec-puppet test.

Say you have a resource with a parameter that is an array, and you only want to validate that the array contains an element.

In our case, say that we have the following code somewhere in our puppet manifests

        host { 'foo.local':
          ip => '1.2.3.4',
          host_aliases => [ 'foo1.local', 'foo2.local' ],
        }

It is pretty direct to validate the exact array inside of an rspec-puppet test

    it { should contain_host('foo.local').with_host_aliases( ['foo1.local', 'foo2.local'] ) }

But that requires a very specific 'expected' value to be passed into the matcher.  What if we just wanted to see if the parameter array includes a given value?  

In a conversation about his problem in the #voxpupuli channel on irc@freenode, dev_el_ops mentioned "one of the design problems of rspec-pupet is that it  doesn't expose property values to the regular rspec matchers"

Since we couldn't get the values directly from the matchers, I had to come up with a different way to approach the problem, which is not quite as readable as I would have wanted.

    it {

        host_aliases = catalogue.resource('host', 'foo.local').send(:parameters)[:host_aliases]

        expect(host_aliases).to include('foo1.local')

    }

Lets break down what is happening.  catalogue.resource('host', 'foo.local') will fetch us the catalog's resource object of type 'host' with the title 'foo.local'.  we can then use 'send' to call the private parameters method on the object to get back it's parameter hash, which we then reference the :host_aliases key to retrieve our array.  

Once we have the array (stored in host_aliases), we can then use a simple 'expect' call with an 'include' matcher to check that the given array includes the value we expect.


Here is a gist of the entire example.

#vDM30in30 - 11


I had some time yesterday, so I finally had the chance to start watching some of the puppet-conf presentations that I did not get a chance to attend personally.. As a reminder, you can find the slides and videos here.

I may morph this into a more 'static' page on my site, and dive in with some thoughts about certain presentations..


Attended -

A Roadmap for a Platform: Mixing Metaphors for Fun and Profit – Eric Sorenson, Puppet (videos) (slides)
Proof of Concept to 30K+ Hosts with Puppet - Petersen Allen, Salesforce (videos) (slides)
Cloud, Containers & the Impact on IT - Jeffrey Snover, Microsoft (videos) (slides)

Puppet Troubleshooting - Thomas Uphill (videos) (slides)
Turning Pain Into Gain - A Unit Testing Story - Nadeem Ahmad & Jordan Moldow (videos)(slides)
Watching the Puppet Show – Sean Porter, Heavy Water Operations (videos) (slides)
Avoiding Toxic Technical Debt Derivatives – R. Tyler Croy, CloudBees, Inc. (videos) (slides)

Using HashiCorp's Vault With Puppet – Seth Vargo, HashiCorp (videos) (slides)
Running Puppet Software in Docker Containers – Gareth Rushgrove, Puppet (videos) (slides)
DevOps Where You Wouldn't Have Expected – Thomas Limoncelli, StackOverflow.com (videos) (slides)
The Future of Testing Puppet Code – Gareth Rushgrove, Puppet (videos) (slides)

Watched -

Service Discovery and Puppet – Marc Cluet, Ukon Cherry (video) (slides)
Enjoying the Journey from Puppet 3.x to 4.x – Rob Nelson, AT&T (videos) (slides)
Collaboration and Empowerment: Driving Change in Infrastructure with Culture – Martin Jackson, Walmart (videos) (slides)
A Look at Looking in the Mirror Actionable Retrospectives - J. Paul Reed (videos) (slides)
External Data in Puppet 4 – R.I. Pienaar (videos) (slides)
Scaling Puppet on AWS ECS with Terraform and Docker – Maxime Visonneau, Trainline (videos) (slides)
Case Study: Puppets in the Government – Kathy Lee (co-author: Glenn Bailey) (videos) (slides)

Want to Watch -

The Truth, Nothing but the Truth: Why Type Systems are Important to Configuration Management – Henrik Lindberg, Puppet (videos) (slides)

Scaling Puppet and Puppet Culture at GitHub - Kevin Paulisse (videos) (slides)
Implementing Puppet within a Complex Enterprise – Jerry Caupain, KPN (videos) (slides)
Moving from Exec to Types and Providers – Martin Alfke, example42 GmbH (videos) (slides)
A Year in Open Source: Automated Compliance With Puppet – Trevor Vaughan, Onyx Point, Inc. (videos) (slides)
Automating Datastore Fleets with Puppet – Joseph Lynch, Yelp (videos) (slides)
Closing the Loop: Direct Change Control with Puppet – Nick Lewis, Puppet (videos) (slides)
Continuous Delivery and DevOps with Jenkins and Puppet Enterprise – Carl Caum, Puppet & Brian Dawson, CloudBees (videos) (slides)
Debugging Diversity – Anjuan Simmons, Assemble Systems (videos) (slides)
Direct Puppet and Application Management for the Puppet Platform – Ryan Coleman, Puppet (videos) (slides)
Puppet 4.x: The Low WAT-tage Edition – Nick Fagerlund, Puppet (videos) (slides)
Puppet Best Practices: Roles & Profiles – Gary Larizza, Puppet (videos) (slides)
Puppet Design Patterns - Lessons From the Gang of Four - David Danzilio (videos) (slides)
Device-Based Modules: Making Them as Simple as a Light Switch – TP Honey, Puppet (videos) (slides)
There is No “I” in DevOps – Bart Driscoll, Dell EMC (videos) (slides)
Writing Custom Types to Manage Web-Based Applications – Tim Cinel, Atlassian (videos) (slides)
Successful Puppet Implementation in Large Organizations – James Sweeny, Puppet (videos) (slides)
Best Practices for Puppet in the Cloud – Randall Hunt, Amazon & Andrew Popp, Servicechannel.com (videos) (slides)
Pulling the Strings to Containerize Your Life - Scott Coulton (videos) (slides)

docker-flow-swarm

#vDM30in30 - 9

I was reading through this week's Docker Weekly, and I saw something that caught my eye... docker-flow.  There was a great video demo about using it.

It serves as a similar role to Registrator, tied to HAProxy, tied with consul template, but directly compatible with docker swarm, and it can bridge to the docker network that you have created to run your swarm containers on.

This looks pretty interesting, and I think I will start using it for my configuration at home, as well as facilitating any demos that I give for our local DevOps Meetup group!

youtube-dl followup - with more docker!

#vDM30in30 - 10

Following up about youtube-dl, I saw in this week's Docker Weekly that someone had a post about running youtube-dl in a container to avoid the messy setup.

https://blog.rphl.io/selfhosted-youtube-downloader-with-docker/

I didn't have to much trouble installing it on my mac, with homebrew, but keeping different applications running in docker (possibly with swarm and docker network bridging/proxying)

youtube-dl to watch youtube offline

#vDM30in30 - 8

youtube-dl -

As you may know, the puppetconf videos, slides, and photos have been released.  I have a friend who is on slow satellite internet with a data-cap, but he is excited about watching the videos.  I have a useful tool installed on my laptop called youtube-dl.  What this allows me to do is point it at the 'playlist' url for all of the videos from puppetconf, and it will download (in parts) all of the videos.

The below commands (and time and bandwidth) are all that I need to pull down all of the videos.  I can resume it if the connection gets interrupted (happened at about 32/94 for me the first day).  

$ mkdir -p ~/puppetconf2016
$ cd ~/puppetconf2016
$ brew install youtube-dl
$ youtube-dl --yes-playlist https://www.youtube.com/playlist?list=PLV86BgbREluVjwwt-9UL8u2Uy8xnzpIqa

After everything is finished downloading, I can transfer them to a USB drive for sharing with friends, and I can watch them while offline.

Hello from the otherside.... (inside the container)

#vDM30in30 - 7

Hello!

I have used ec2 tags to gather information into facter, and I had wanted to duplicate that same kind of logic using docker containers and labels... Unfortunately it seems like labels are 'external' only to the container.
I found several issues which all end up pointing at this 'introspection' issue.
https://github.com/docker/docker/issues/8427

It looks like the only way currently is to have some sort of external service manage that information for you (etcd/consul/mesos/pass it in via environment variables to your container / kubernettes downward-api) OR to mount the docker socket inside your container and go through the json output (not very secure)

Reasons for getting that information seem to range from enabling service discovery (knowing what actual host/IP you are bound to) to knowing what size JVM heap args to set inside a container based on actual container run environment.

It seems like memory information was added with https://github.com/docker/docker/pull/13312/files which allows a readonly mount of the /sys/fs/cgroup location specific to the container...

$ docker run --rm -it --entrypoint /bin/bash puppet/puppet-agent-ubuntu

root@96acc60738a7:/# cat /sys/fs/cgroup/memory/memory.max_usage_in_bytes
19025920
root@96acc60738a7:/# exit
exit
$ docker run -m 2048m --rm -it --entrypoint /bin/bash puppet/puppet-agent-ubuntu
WARNING: Your kernel does not support swap limit capabilities, memory limited without swap.
root@dab851113412:/#  cat /sys/fs/cgroup/memory/memory.max_usage_in_bytes
2654208
root@dab851113412:/#

Find non-utf8 characters in your puppet code!

#vDM30in30 - 6

I was checking out the #puppet channel on irc this morning, and someone came in asking for help with the following error..

err: Could not retrieve catalog from remote server: Error 400 on SERVER: (<unknown>): control characters are not allowed at line 1 column 1

I couldn't find anything puppet specific, but I found several posts referencing sidekiq and having that error, and it seemed to be due to the presence of non UTF-8 characters in their files/code.

I found a few answers on stackoverflow for detecting non UTF-8 chars..

http://stackoverflow.com/a/9395552 or http://stackoverflow.com/a/13702856

And to add in the 'search hiera and puppet and erb' files find option.  Here is the 'exclude chars we thing are not in range' command

find $CODEBASE/ -type f \( -name '*.pp' -o -name '*.yaml' -o -name '*.erb' \) | xargs grep --color='auto' -P -n "[\x80-\xFF]"

Here is the 'exclude chars that do not match the expected range' command

find $CODEBASE/ -type f \( -name '*.pp' -o -name '*.yaml' -o -name '*.erb' \) | xargs grep --color='auto' -P -n "[^\x00-\x7F]"

This is something that you could even add as a validation check to your CI system to prevent you from checking in invalid characters to your puppet codebase.

Roles and Profiles - an official example

#vDM30in30 #5

After first learning about the roles and profiles pattern, I spent quite a while searching long and hard for some good examples of the pattern.

Today, I stumbled onto some official docs in the PE 2016.4 pages that give a well thought out example of setting up jenkins masters, and the refactoring process that led them to the profile that they use now.



https://docs.puppet.com/pe/2016.4/r_n_p_intro.html
https://docs.puppet.com/pe/2016.4/r_n_p_full_example.html
https://docs.puppet.com/pe/2016.4/r_n_p_profiles.html