Hi everyone!
Anyone who knows me, knows that I listen to a LOT of podcasts. In an attempt to blog more regularly, and to be able to search for things that 'I know I heard somewhere!' I'm going to try to keep track of things of interest in podcasts and blogposts that I've read. If I I can keep active enough with this, I may try rolling it up into a weekly feature!
SE-Radio #225: Monicia Beckwith on Java Garbage detection.
This episode talked about the history of java garbage collection algorithms. I found the discussion on generational garbage collection really fascinating. That was how copying young objects back and forth performs automatic compaction. You also can 'age' the data, since your program typically has two types of memory.. quickly allocated and abandoned objects and long lived objects.
There was some discussion about GC usage and optimizations, about how some systems with really tight 'worst case' latency requirements actually will have a clustered design, and take nodes offline and manually initiate their really slow, but full heap garbage collection while the other nodes are still online and accepting traffic. There were some command line flags mentioned, but I was mowing and didn't write them down, but they are useful for looking at some of the really simple information put out by the GC, and you can view them with histogram tools to analyze your usage patterns in preparation for tuning.
Things to ponder... The GC patterns are dependent on your application and it's usage patterns.. I wonder how the signature of garbage collection changes when you are using immutable data structures, and some of the other languages that run on the JVM?
Partially Derrivative: S2E8: The Love child of Princess Leia and Jabba the Hutt
They talked about using Machine Learning and Data Science to categorize StarWars spoilers and help keep people from seeing them. The persons work was made available as a chrome extension..
Also, they had a segment where they talked with Micheal Kennedy of Talk Python to me about Software Engineering tips, and he gave a discussion on Generators and Co-routines. This is a topic I've heard about, but need to get more in-depth with. I've been hearing lots of episodes talking about different functional programming techniques, and need to start seeing how I can incorporate them into some of the things I do each day.
Things to Ponder... He gave a list of languages which have concepts like generators and co-routines... Do any of the langues I use have them? How could it be applied?
Showing posts with label Java. Show all posts
Showing posts with label Java. Show all posts
Saturday, April 30, 2016
Tuesday, March 3, 2015
What's the diff?
Hello everyone!
Have you ever had to compare two files? It's pretty easy on linux, there is a nice standard tool called 'diff' which can show you the differences between two text files. you also can use vimdiff to get a nicer way of seeing the differences, word by word instead of just line by line...
But what do you do if you need to diff two jar files? or if you want to diff two binary files while also displaying any strings that happen to be present?
Behold, a nice script that will let you use almost any tool on your path to output a representation of, well, ANYTHING, and then diff it with the tool of your choice. I may end up developing a version that could take different command line arguments to change things up, but right now I just edit the file and save it with different names, like jardiff.pl, binarydiff.pl, etc...
The $diff_tool is pretty self explanatory... you can use diff, vimdiff, or whatever diff program you have available.
To make sure we execute this with bash, we use bash -c \" $CMD_HERE \" to execute the command. We need to escape the quotes because we quoted the string to assign it to my $cmd.
The $diff_tool is pretty simple, we just envoke the command we specified earlier. In most cases, your diff tool expects to get two files to compare (you can use a tool like diff3 to do a three-way diff).
This brings us to the <( ... )operator. This executes the given command in a subshell, and output gets passed in as a file handle back to the original command. If we didn't do this, we would have to dump the output of each of the individual $vis_tool commands to a temp file, and then pass each file to the diff tool, and then clean up our temp files.
Finally, we actually execute the command with
This diff technique has been very helpful for me comparing all kinds of files, and I hope it can help you too!
Have you ever had to compare two files? It's pretty easy on linux, there is a nice standard tool called 'diff' which can show you the differences between two text files. you also can use vimdiff to get a nicer way of seeing the differences, word by word instead of just line by line...
But what do you do if you need to diff two jar files? or if you want to diff two binary files while also displaying any strings that happen to be present?
Behold, a nice script that will let you use almost any tool on your path to output a representation of, well, ANYTHING, and then diff it with the tool of your choice. I may end up developing a version that could take different command line arguments to change things up, but right now I just edit the file and save it with different names, like jardiff.pl, binarydiff.pl, etc...
jardiff.pl
#!/usr/bin/perl -w use strict; use warnings; use diagnostics; my $left = shift(); my $right = shift(); die unless ((-e $left) and (-e $right)); my $vis_tool = "jar -tf"; my $diff_tool = "vimdiff"; my $cmd="bash -c \"$diff_tool <($vis_tool $left) <$vis_tool $right)\""; exit system($cmd);Lets go over what is going on with each line...
#!/usr/bin/perl -w use strict; use warnings; use diagnostics;A good way to start out perl scripts, first line tells your shell what program to execute, and the rest of the lines are good ways of checking yourself if you make a mistake.. giving useful parsing error messages, enforcing stricter syntax, etc..
my $left = shift(); my $right = shift();This brings the 1st and 2nd arguments off the command line and stores them in the $left and $right values.
die unless ((-e $left) and (-e $right));
This line exits the script if both the left and right arguments do not exist as files on your system. I should add a usage statement to make things easier.
my $vis_tool = "jar -tf"; my $diff_tool = "vimdiff";Here is where our choices come in. In this instance, our visualization tool is 'jar -tf', which lists the contents of a jar specified by the file you pass to the command. If you also care about the dates and timestamps, you can add the -v command to also display that additional information.
The $diff_tool is pretty self explanatory... you can use diff, vimdiff, or whatever diff program you have available.
my $cmd="bash -c \"$diff_tool <($vis_tool $left) <$(vis_tool $right)\""; exit system($cmd);And this simply puts all the parts together, and runs it on the shell. Lets work left to right.
To make sure we execute this with bash, we use bash -c \" $CMD_HERE \" to execute the command. We need to escape the quotes because we quoted the string to assign it to my $cmd.
The $diff_tool is pretty simple, we just envoke the command we specified earlier. In most cases, your diff tool expects to get two files to compare (you can use a tool like diff3 to do a three-way diff).
This brings us to the <( ... )operator. This executes the given command in a subshell, and output gets passed in as a file handle back to the original command. If we didn't do this, we would have to dump the output of each of the individual $vis_tool commands to a temp file, and then pass each file to the diff tool, and then clean up our temp files.
Finally, we actually execute the command with
exit system($cmd);The other really useful way that I make this script is my 'binary' diff version. Most people would use hexdump to get a more diff-able version of a binary file, but I actually prefer to use xxd to get output that is both in hex and it attempts to 'stringify' as much of the binary as it can, and it shows it off to the side.
daryl$ xxd xpp3_min-1.1.4c.jar | head0000000: 504b 0304 0a00 0000 0000 12a5 6a35 0000 PK..........j5.. 0000010: 0000 0000 0000 0000 0000 0900 0000 4d45 ..............ME 0000020: 5441 2d49 4e46 2f50 4b03 040a 0000 0008 TA-INF/PK....... 0000030: 0011 a56a 35e5 823f 7b5e 0000 006a 0000 ...j5..?{^...j.. 0000040: 0014 0000 004d 4554 412d 494e 462f 4d41 .....META-INF/MA 0000050: 4e49 4645 5354 2e4d 46f3 4dcc cb4c 4b2d NIFEST.MF.M..LK- 0000060: 2ed1 0d4b 2d2a cecc cfb3 5230 d433 e0e5 ...K-*....R0.3.. 0000070: 72cc 4312 712c 484c ce48 5500 8a01 25cd r.C.q,HL.HU...%. 0000080: f48c 78b9 9c8b 5213 4b52 5374 9d2a 41ea ..x...R.KRSt.*A. 0000090: 4df5 0ce2 0d8c 7593 0ccc 1534 824b f314 M.....u....4.K..
This diff technique has been very helpful for me comparing all kinds of files, and I hope it can help you too!
Saturday, February 14, 2015
Weblogic Identity Asserter and Athorization Provider in one!
When trying to create a Custom Identity Asserter AND Authorization Provider MBean for weblogic, I ran into some trouble.. After following all of the examples, I was unable to get my MBean to behave as both!
While trying to find solutions, my googleFu turned up a few lead that left me feeling like it could not be done.
http://www.ateam-oracle.com/why-do-i-need-an-authenticator-when-i-have-an-identity-asserter/
https://community.oracle.com/thread/796040
The problem I had was if my MDF file extended IdentityAsserter, it would not contain a method to 'getControlFlag' to pass to the LoginModule.
I found this great example at http://danielveselka.blogspot.com/2011/10/weblogic-custom-identity-asserter.html. The only problem? He skirts around the same issue by manually assigning a constant value to the controlFlag that gets passed to the AppConfigurationEntry in the getConfiguration() method.
In that example, he has the following in the MDF. If I had this, My MBean would not build with the getControlFlag() method.
If I extended the Authenticator, my assertIdentity() methods from the IdentityAsserter would never get called, but I WOULD have the getControlFlag() method
Turns out this is an easy fix. There is an attribute value for Implements in the MDF Element Syntax. Having an MBeanType similar to the following solved my problem!
This allows it to implement all the features of both, but be a single entry that you can add to your weblogic security realm.
While trying to find solutions, my googleFu turned up a few lead that left me feeling like it could not be done.
http://www.ateam-oracle.com/why-do-i-need-an-authenticator-when-i-have-an-identity-asserter/
https://community.oracle.com/thread/796040
The problem I had was if my MDF file extended IdentityAsserter, it would not contain a method to 'getControlFlag' to pass to the LoginModule.
I found this great example at http://danielveselka.blogspot.com/2011/10/weblogic-custom-identity-asserter.html. The only problem? He skirts around the same issue by manually assigning a constant value to the controlFlag that gets passed to the AppConfigurationEntry in the getConfiguration() method.
public void initialize(ProviderMBean mbean, SecurityServices services)
{
System.out.println("SimpleSampleIdentityAsserterProviderImpl.initialize");
SimpleSampleIdentityAsserterMBean myMBean = (SimpleSampleIdentityAsserterMBean)mbean;
description = myMBean.getDescription() + "\n" + myMBean.getVersion();
controlFlag = LoginModuleControlFlag.SUFFICIENT;
...
} private AppConfigurationEntry getConfiguration(HashMap options)
{
System.out.println("SimpleSampleIdentityAsserterProviderImpl: getConfiguration");
// make sure to specify the simple sample authenticator's login module
// and to use the control flag from the simple sample authenticator's mbean.
return new
AppConfigurationEntry(
"examples.security.providers.authentication.simple.SimpleSampleLoginModuleImpl",
controlFlag,
options
);
}In that example, he has the following in the MDF. If I had this, My MBean would not build with the getControlFlag() method.
Extends = "weblogic.management.security.authentication.IdentityAsserter"If I extended the Authenticator, my assertIdentity() methods from the IdentityAsserter would never get called, but I WOULD have the getControlFlag() method
Extends = "weblogic.management.security.authentication.Authenticator"Turns out this is an easy fix. There is an attribute value for Implements in the MDF Element Syntax. Having an MBeanType similar to the following solved my problem!
<MBeanType Name = "SimpleSampleIdentityAsserter" DisplayName = "SimpleSampleIdentityAsserter" Package = "examples.security.providers.identityassertion.simple" Extends = "weblogic.management.security.authentication.IdentityAsserter"Implements = "weblogic.management.security.authentication.Authenticator"PersistPolicy = "OnUpdate" >
This allows it to implement all the features of both, but be a single entry that you can add to your weblogic security realm.
Thursday, January 8, 2015
Thoughts on podcasts...
Did you ever feel stuck in a rut? I sure did!
In 2012, I realized I was not growing at my job. I had been there for 5 years, but I felt like I had 5 first years of experience. My skills were atrophying and I was not growing as a programmer. I was also putting on a bit of weight (Up to 220 lbs).
No one was going to help me improve but me, so I decided to get both my coding knowledge and my body into shape. I started riding to work (5.5 miles each way) and listening to podcasts while I was riding. I also began tracking down a bunch of technical blogs and adding them to my Google Reader.
It was difficult to track down technical podcasts that had useful information for me at my particular point in my journey; Computer Engineering grad, 5 years of dev experience with Java, just discovered TDD. I had been mostly listening to NPR's Radiolab.
I eventually stumbled onto Ruby Rogues. I listened to a few episodes that seemed to have language agnostic topics... This began to open my eyes to some of the current state of the art ideas that I had been missing. Eventually, I just started listening to all of the episodes, and then stumbled onto JavaScript Jabber. Similar layout, some more relevant things (I have done a bit of JavaScript at my job).
I had been following Alan Ashcraft's Morning Dew Drop, where I found several podcasts that were starting to pique my interest... (The Changelog, Walled Garden Weekly), but eventually saw a link to getupandcode.com. This was a great podcast... focused on fitness for programmers, exactly what I needed! Though listening to getupandcode, I saw that John Sonmez compiled The Ultimate List of Developer Podcasts and found a few more on there!
I had found SoftwareEngineering Radio, but the episodes were very long and mostly didn't keep my attention. I stumbled onto Java Posse. This podcast seemed like the best overall fit for what I had been looking for - Java specific practices and concepts, enterprise focus, and not to dry! I heard them talking about Puppet, Chef, and other DevOps concepts and thought it sounded like a pretty amazing place to be. I eventually found some references to some DevOps concepts, and found DevOps Mastery and DevOps Cafe.
While bouncing around between podcasts, I heard Saron on RubyRogues, and that she was starting up a site/twitter chat/podcast called CodeNewbie.... only one problem, I didn't have Twitter! I started up a twitter account, and began participating in the twitter chat.
Though the CodeNewbie shoutouts, I found another very interesting podcast.... This Developer's Life. Like all podcasts, I listened to an episode, liked it, and started back from the beginning.
Where does that leave me now? I've started this blog, and I've been bothering my co-workers over and over with all of the interesting things that I've been exposing myself to. I figured I would try to start putting up posts about individual episodes... either going back and re-listening to some that really stood out, or just commenting on new ones as they air.
What did I gain from all of my podcast listening? I've been exposed to all kinds of 'state of the art' ideas and tech, I had someething to do on my bike commute (which brought me down to ~155 lbs), and it helped me rediscover my desire to learn new things! What it DIDN'T do was help me get any side projects done really leveraging these new ideas and tools that I was hearing about! Hearing is no replacement for doing, so I'm going to try to do some more hands-on diving into frameworks and languages.
Are there any podcasts that you enjoy listening to? Leave a note below!
In 2012, I realized I was not growing at my job. I had been there for 5 years, but I felt like I had 5 first years of experience. My skills were atrophying and I was not growing as a programmer. I was also putting on a bit of weight (Up to 220 lbs).
No one was going to help me improve but me, so I decided to get both my coding knowledge and my body into shape. I started riding to work (5.5 miles each way) and listening to podcasts while I was riding. I also began tracking down a bunch of technical blogs and adding them to my Google Reader.
It was difficult to track down technical podcasts that had useful information for me at my particular point in my journey; Computer Engineering grad, 5 years of dev experience with Java, just discovered TDD. I had been mostly listening to NPR's Radiolab.
I eventually stumbled onto Ruby Rogues. I listened to a few episodes that seemed to have language agnostic topics... This began to open my eyes to some of the current state of the art ideas that I had been missing. Eventually, I just started listening to all of the episodes, and then stumbled onto JavaScript Jabber. Similar layout, some more relevant things (I have done a bit of JavaScript at my job).
I had been following Alan Ashcraft's Morning Dew Drop, where I found several podcasts that were starting to pique my interest... (The Changelog, Walled Garden Weekly), but eventually saw a link to getupandcode.com. This was a great podcast... focused on fitness for programmers, exactly what I needed! Though listening to getupandcode, I saw that John Sonmez compiled The Ultimate List of Developer Podcasts and found a few more on there!
I had found SoftwareEngineering Radio, but the episodes were very long and mostly didn't keep my attention. I stumbled onto Java Posse. This podcast seemed like the best overall fit for what I had been looking for - Java specific practices and concepts, enterprise focus, and not to dry! I heard them talking about Puppet, Chef, and other DevOps concepts and thought it sounded like a pretty amazing place to be. I eventually found some references to some DevOps concepts, and found DevOps Mastery and DevOps Cafe.
While bouncing around between podcasts, I heard Saron on RubyRogues, and that she was starting up a site/twitter chat/podcast called CodeNewbie.... only one problem, I didn't have Twitter! I started up a twitter account, and began participating in the twitter chat.
Though the CodeNewbie shoutouts, I found another very interesting podcast.... This Developer's Life. Like all podcasts, I listened to an episode, liked it, and started back from the beginning.
Where does that leave me now? I've started this blog, and I've been bothering my co-workers over and over with all of the interesting things that I've been exposing myself to. I figured I would try to start putting up posts about individual episodes... either going back and re-listening to some that really stood out, or just commenting on new ones as they air.
What did I gain from all of my podcast listening? I've been exposed to all kinds of 'state of the art' ideas and tech, I had someething to do on my bike commute (which brought me down to ~155 lbs), and it helped me rediscover my desire to learn new things! What it DIDN'T do was help me get any side projects done really leveraging these new ideas and tools that I was hearing about! Hearing is no replacement for doing, so I'm going to try to do some more hands-on diving into frameworks and languages.
Are there any podcasts that you enjoy listening to? Leave a note below!
Subscribe to:
Posts (Atom)