Ryan's Inter-Online Web Blog - Latest comments on imap2maildir: a tool for mirroring IMAP to maildir http://blog.hoopycat.com/?disp=comments en-US http://backend.userland.com/rss 60 Ryan [Member] in response to: imap2maildir: a tool for mirroring IMAP to maildir Tue, 29 Jun 2010 13:47:39 +0000 Ryan [Member] c624@http://blog.hoopycat.com/ Thanks! :-)<br /> <br /> I read last night about <a href="https://wiki.ubuntu.com/Quickly">Quickly</a> in the latest <a href="http://www.linuxjournal.com/">Linux Journal</a>... it's a framework for something the article referred to as "opportunistic programming." Putting a name on, and giving respect to, my style and motivation for programming was pretty awesome. (The framework looks pretty awesome, too.)<br />
I read last night about Quickly in the latest Linux Journal... it's a framework for something the article referred to as "opportunistic programming." Putting a name on, and giving respect to, my style and motivation for programming was pretty awesome. (The framework looks pretty awesome, too.)
]]> http://blog.hoopycat.com/2009/07/imap2maildir-a-tool-for-mirroring-imap-t#c624 Miblon [Visitor] in response to: imap2maildir: a tool for mirroring IMAP to maildir Sat, 26 Jun 2010 22:01:21 +0000 Miblon [Visitor] c618@http://blog.hoopycat.com/ Excelent work and way to go! The best open source products evolved because of people needing it and by putting your code on github you created an good starting point!<br /> <br /> I will give it a spin later next week or so. I intend to have it running on one of my own servers that then backups to another one of my servers :D. I am a bit of a python programmer myself, so if I come across something I think can help enhance your code, you will find it in github too.<br /> <br /> Respect!<br /> <div class="postsnapdiv"> <p class="postsnapptag">On Oct 22, Miblon wrote "<a href="http://www.dogodigi.net/node/24" title="read milo's blog post">asus eee 1005HA-H met moblin 2.1</a>".</p></div>
I will give it a spin later next week or so. I intend to have it running on one of my own servers that then backups to another one of my servers :D. I am a bit of a python programmer myself, so if I come across something I think can help enhance your code, you will find it in github too.

Respect!

On Oct 22, Miblon wrote "asus eee 1005HA-H met moblin 2.1".

]]> http://blog.hoopycat.com/2009/07/imap2maildir-a-tool-for-mirroring-imap-t#c618 jd [Visitor] in response to: imap2maildir: a tool for mirroring IMAP to maildir Sat, 15 Aug 2009 13:43:13 +0000 jd [Visitor] c278@http://blog.hoopycat.com/ Sent you a msg via gmail regarding your recent changes. http://blog.hoopycat.com/2009/07/imap2maildir-a-tool-for-mirroring-imap-t#c278 Ryan [Member] in response to: imap2maildir: a tool for mirroring IMAP to maildir Fri, 14 Aug 2009 12:38:02 +0000 Ryan [Member] c276@http://blog.hoopycat.com/ OK, I've reworked a lot of the IMAP handling stuff in the newiterator branch, and am able to process my mailbox of great girth (92,470 messages) with no abnormal memory issues.<br /> <br /> It is slower, alas. It took about 10 hours on my mailbox (!), so I've added some additional caching and verification of the local files (aka "turbo mode"). This has reduced the time on my mailbox down to a less-unreasonable ~3 hours.<br /> <br /> I think I might be able to do better, but this is a huge mailbox, and that's why we have cron, right?<br /> <br /> On a 192-message mailbox:<br /> <br /> master branch: 5.0 seconds<br /> newiterator branch, no turbo mode: 33.0 seconds<br /> newiterator branch, turbo mode: 2.1 seconds<br /> <br /> But more importantly: no memoryerror crash!<br /> <br /> I'll probably merge newiterator into master early next week, unless something bursts into flame... -rt
It is slower, alas. It took about 10 hours on my mailbox (!), so I've added some additional caching and verification of the local files (aka "turbo mode"). This has reduced the time on my mailbox down to a less-unreasonable ~3 hours.

I think I might be able to do better, but this is a huge mailbox, and that's why we have cron, right?

On a 192-message mailbox:

master branch: 5.0 seconds
newiterator branch, no turbo mode: 33.0 seconds
newiterator branch, turbo mode: 2.1 seconds

But more importantly: no memoryerror crash!

I'll probably merge newiterator into master early next week, unless something bursts into flame... -rt]]> http://blog.hoopycat.com/2009/07/imap2maildir-a-tool-for-mirroring-imap-t#c276 jd [Visitor] in response to: imap2maildir: a tool for mirroring IMAP to maildir Sat, 08 Aug 2009 23:03:53 +0000 jd [Visitor] c275@http://blog.hoopycat.com/ Looking forward to your changes...<br /> <br />
]]> http://blog.hoopycat.com/2009/07/imap2maildir-a-tool-for-mirroring-imap-t#c275 Ryan [Member] in response to: imap2maildir: a tool for mirroring IMAP to maildir Sat, 08 Aug 2009 12:16:35 +0000 Ryan [Member] c274@http://blog.hoopycat.com/ I have noticed that on my end, too. I've been working on a number of major changes in a new branch, which should help with memory performance... not yet ready for prime time (and it kinda took the back seat to some other projects recently), but I think it will be better in the long run.<br /> <br /> Glad to know it at least tries to work under Windows! :-)<br /> <br /> The new branch is at: http://github.com/rtucker/imap2maildir/tree/newiterator
Glad to know it at least tries to work under Windows! :-)

The new branch is at: http://github.com/rtucker/imap2maildir/tree/newiterator]]> http://blog.hoopycat.com/2009/07/imap2maildir-a-tool-for-mirroring-imap-t#c274 jd [Visitor] in response to: imap2maildir: a tool for mirroring IMAP to maildir Fri, 07 Aug 2009 18:30:29 +0000 jd [Visitor] c273@http://blog.hoopycat.com/ Thanks for sharing this.<br /> <br /> I was able to grab a few messages from Gmail, but eventually I get a MemoryError in Python's imaplib, on line 1150:<br /> <br /> data = self.sslobj.read(size-read)<br /> <br /> I understand this isn't your code but do you have any ideas on how to fix this? This is using Python 2.5 (on Windows, but that shouldn't matter).<br /> <br /> Thanks...<br />
I was able to grab a few messages from Gmail, but eventually I get a MemoryError in Python's imaplib, on line 1150:

data = self.sslobj.read(size-read)

I understand this isn't your code but do you have any ideas on how to fix this? This is using Python 2.5 (on Windows, but that shouldn't matter).

Thanks...
]]> http://blog.hoopycat.com/2009/07/imap2maildir-a-tool-for-mirroring-imap-t#c273