Memory Leak

When we want to iterate a long list, we can simply write a query and get a cursor, ActiveRecord will do all the heavy lifting for us. What happens when we need to do some complicated computations on a set of data, which sometimes can be too big to be stored in memory for the entire computing process? This is when we need to start being more creational. I’d like to introduce what I came up with. The problem: - Complex calculation on time based data series for a period of 3 months. - Each calculation may depends on previous one and on future and past data. - Must be in order. - When fetching all data server crash on memory. The solution: I wanted to do the most minor code change possible, and currently the data was accessed via a hash. I decided to encapsulate the hash with something I called lazy chunked hash (tried google it see it as standard behavior in clojure). It looks like this: class ValuesProvider def initialize() @loaded_date = nil @hash = Hash.new(0) end def [](time_slot) get(time_slot) end private def get(time_slot) relevant_date = time_slot.to_date unless relevant_date == @loaded_date load(relevant_date) end @hash[time_slot.to_i] endend Pretty simple and does the work, instead of loading the data all at once, the data is being loaded for each day separately, this way we keep it chunky but not too chunky. And best part, my code that consume the data, didn’t change because of the [] method, which makes my ValueProvider behave like an array. This solution is good when the consumer data request(call for[]) implies on what data should be loaded, which most of the times will, but in some cases it won’t)

I found this only today, you should run Rubymine as 32 bit process. It works much faster and consume half the memory. To set this got to the Applications folder right click on rubymine->get info and check the “open in 32 bit mode” That’s all!

Today I ran into a guy who worked with me on my previous project. On that project I was responsible to rewrite the WCF channels (proxies) generation management and lifetime management of them. I decided to do some research and found some good stuff here. So this guy (aka Kurki) tells me they found a memory leak in my manager, what was the problem? So i messed up like I always do (well, only 90% of the time, like a good programmer). I wrote a code that registers to the Faulted event of the channel, kills the channel when Faulted and creates a new one to work with, everything looked fine, only one screw up (I’m sure some of you can guess). I forgot to unregister from the Faulted Event. On the web they tell you to do some unique logic to kill your channel in a safe way, what they don’t talk about is that it’ll stay in memory and fire events if you won’t unregister from stuff although you think it is dead. So please do not forget to unregister from stuff even if the object is disposable. Or consider the use of weak event listeners and problem solved.