This week I had to do a few installations of Puppet Enterprise on an EC2 instance. Although this seemed like a simple enough task, I did hit upon one annoying issue.

I was doing a Monolithic installation as it was just for testing, I had done this several times on Digital Ocean↗ and locally using a Vagrant box↗ with no issue at all. Reading through the documenation I had to open the following

8140

  • The Puppet master uses this port to accept inbound traffic/requests from Puppet agents.
  • The PE console sends request to the Puppet master on this port.
  • Certificate requests are passed over this port unless ca_port is set differently.

443

  • This port provides host access to the PE console.
  • The PE Console accepts HTTPS traffic from end-users on this port.

61613

  • MCollective uses this port to accept inbound traffic/requests from Puppet agents for orchestration.
  • Any host used to invoke orchestration commands must be able to reach MCollective on this port.

That seemed simple enough, however when I create a security group with those ports open to world, also as I was using the web based installer I opened port 3000 to my IP address, just in-case come script kiddie decided to do the installation for me.

Everything ran as expected until the installation got to “Waiting for Node Classifier to start” and then it hung;

Puppet Enterprise & AWS Security Groups 1/2
PWuapiptHHHHHeiTTTTTtnTTTTTDgPPPPPB_____fPPPPPcoRRRRRorOOOOOnXXXXXfNYYYYYio=====gduehhhhhrttttteCtttttdlppppp.a_____spppppsrrrrriooooofxxxxxiyyyyye=====rHHHHHtTTTTToTTTTTPPPPPsSSSSSt_____aPPPPPrRRRRRtOOOOO…XXXXXYYYYY=====hhhhhttttttttttpppppsssss_____ppppprrrrroooooxxxxxyyyyy=====opppppttttt/////pppppuuuuuppppppppppeeeeetttttlllllaaaaabbbbbsssss/////pppppuuuuuppppppppppeeeeettttt/////bbbbbiiiiinnnnn/////cccccuuuuurrrrrlllll—————tttttlllllsssssvvvvv11111-----sssss—————cccccaaaaaccccceeeeerrrrrttttt/////eeeeetttttccccc/////pppppuuuuuppppppppppeeeeetttttlllllaaaaabbbbbsssss/////pppppuuuuuppppppppppeeeeettttt/////sssssssssslllll/////ccccceeeeerrrrrtttttsssss/////cccccaaaaa.....pppppeeeeemmmmm—————kkkkkeeeeeyyyyyopppppttttt/////pppppuuuuuppppppppppeeeeetttttlllllaaaaabbbbbsssss/////ssssseeeeerrrrrvvvvveeeeerrrrr/////dddddaaaaatttttaaaaa/////cccccooooonnnnnsssssooooollllleeeee-----ssssseeeeerrrrrvvvvviiiiiccccceeeeesssss/////ccccceeeeerrrrrtttttsssss/////pppppuuuuuppppppppppeeeeettttt.....mmmmmccccckkkkkeeeeennnnndddddrrrrriiiiiccccckkkkk.....iiiiiooooo.....ppppprrrrriiiiivvvvvaaaaattttteeeee_____kkkkkeeeeeyyyyy.....pppppeeeeemmmmm—————ccccceeeeerrrrrtttttopppppttttt/////pppppuuuuuppppppppppeeeeetttttlllllaaaaabbbbbsssss/////ssssseeeeerrrrrvvvvveeeeerrrrr/////dddddaaaaatttttaaaaa/////cccccooooonnnnnsssssooooollllleeeee-----ssssseeeeerrrrrvvvvviiiiiccccceeeeesssss/////ccccceeeeerrrrrtttttsssss/////pppppuuuuuppppppppppeeeeettttt.....mmmmmccccckkkkkeeeeennnnndddddrrrrriiiiiccccckkkkk.....iiiiiooooo.....ccccceeeeerrrrrttttt.....pppppeeeeemmmmmhhhhhttttttttttpppppsssss::::://pppppuuuuuppppppppppeeeeettttt.....mmmmmccccckkkkkeeeeennnnndddddrrrrriiiiiccccckkkkk.....iiiiiooooo:::::44444444443333333333/////ccccclllllaaaaassssssssssiiiiifffffiiiiieeeeerrrrr-----aaaaapppppiiiii/////v11111/////lllllaaaaasssssttttt-----ccccclllllaaaaassssssssss-----uuuuupppppdddddaaaaattttteeeeegggggrrrrreeeeeppppp-----qqqqqlllllaaaaasssssttttt_____uuuuupppppdddddaaaaattttteeeee.....*****[[[[[[[[[[:::::dddddiiiiigggggiiiiittttt:::::]]]]]]]]]]

I tried running the curl command manually and got the same problem;

Puppet Enterprise & AWS Security Groups 2/2
opt/puppetlabs/puppet/bin/curl—tlsv1-s—cacert/etc/puppetlabs/puppet/ssl/certs/ca.pem—keyopt/puppetlabs/server/data/console-services/certs/puppet.mckendrick.io.private_key.pem—certopt/puppetlabs/server/data/console-services/certs/puppet.mckendrick.io.cert.pemhttps://puppet.mckendrick.io:4433/classifier-api/v1/last-class-update

It was obviously an issue with the security group as when I added an allow all rule and re-ran the installer it worked without issue, however after the installation completed I removed the allow all rule and I could no longer login to the Puppet Enterprise Dashboard.

After repeating the installation a few times I managed to track the issue down to the way that Puppet interacts with itself. To resolve the issue I had to add an allow all for both external IP address and the internal IP address.

While this feels like a bit of a fudge it solved the issue with both the installer and the dashboard.

If anyone knows a more elegant fix for this please let me know in the comments below.