Yesterday's news

Today Fabio announced that NHibernate 3.0 General Availability can be downloaded from its home at Source Forge.

Here is a list of most notable (IMHO) features:

• New, full-featured, and extensible, Linq to NHibernate implementation (LINQ2NH 1.0 used to be a simple wrapper around ICriteria API):
  • A blog post on the official NHForge blog announcing availability of LINQ2NH 2.0
  • A blog post on Fabio’s blog explaining how to extend LINQ2NH with your own custom logic 
  • This LINQ2NH 2.0 slide deck from recent NHibernate Day conference in Europe provides some interesting details on capabilities and limitations
• The new QueryOver API. Positioned as a compromise between ICriteria API and LINQ, this strongly-typed fluent API gives much better control over generated SQL than what is possible with LINQ:
  • QueryOver in NHibernate 3.0 on NHForge blog
• Customizable SQL exception translation can now coexist with batching enabled – catch SQL errors (FK violations, unique constraints etc) and convert to your own Exception types:
  • Fabio Maulo on Improving NHibernate ADO.NET Exception Management
  • NH-2020
• Compatibility with ConfORM, the new fluent-style mapping engine similar to Fluent NHibernate. While Fluent NH mappings are supposed to be written by hand by the developer, ConfORM originated from an effort to generate NH mappings from XSDs. Unlike Fluent NH, ConfORM does not generate HBM.XML mappings behind the scene. Instead it uses the new NH 3.0 mapping extension points to inject mappings “directly” into Session Factory.
  • Fabio’s blog post explaining motivation behind ConfORM
  • Stephen Bohlen speculating about potential advantages of the no-XML approach
  • ConfORM home on Google Code
  • A blog post by Patrick Weibel trying out ConfORM 
• Improved “fluent” SessionFactory configuration:
  • Fabio Maulo on NHibernate configuration Part 1 
  • Part 2
  • Part 3 (configuring second-level cache)
• Deferred loading of columns (aka “lazy properties“) — useful when your entity has a huge text column that is used rarely:
  • Ayende on NHibernate 3.0 Lazy Properties
  • NH-429
• The new pluggable logging subsystem (NHibernate.Logging) removes hard dependency on log4net. Now you can use CommonLogging or NLog - NH-1554
• Improved support for storing UTC dates in your database - NH-1135
• Support for memcached 1.4.4 and newer - NHCH-27 - the contribution I’m proud of!
• Experimental support of the new and much improved memcached client library called “Enyim” - finally you can leverage all features of newer Memcached implementations such as Membase:
  • NHCH-26 aka NHibernate.Caches.EnyimMemcached
  • Enyim Memcached Client official site

Misc Notes:

• NHibernate 3.0 is now compiled against .NET 3.5 Client Profile (but is known to work with 4.0 just fine)
• Please note that it may take a few days for the sources to get uploaded to the SymbolSource for your Visual Studio step-by-step debugging convenience
• Corresponding GA builds of other NHContrib projects (Caches, Validator etc.) are being released over the next day or two
• For full list of improvements and bug fixes see official Release Notes

(Inspired by a security.stackexchange.com question)

Locking the account after 3 attempts may seem like a good security measure but it may be very perplexing for the user. Having secret questions as a way to unlock the account is a hole in security. Having the admins having to unlock the accounts does not scale well.

If you think about it, the only reason we put the lockout in place is to prevent the brute force password attacks.

Here are the new guidelines from Bill Cheswick of AT&T at OWASP AppSec 2010 conference that are designed to address these issues:

• Don’t count the same invalid password reentered multiple times in a row towards the lockout limit;
• Instead of having users setup secret question which effectively is a (weak) secondary password, have them setup password hint that would remind them what the primary password is;
• Lock the account temporarily, in increasing time increments: e.g. 5, 10, 20, 60 mins

SVN Switch is a command you use to switch your working copy between the multiple branches of your code. Basically, it deletes all the extra stuff, downloads all the missing stuff, and updates all the common stuff.

Switching from a pristine clean checkout always works without error. However, as explained in SVN FAQ, in some cases switch stops, leaving the working copy half-switched. This happens if there are unversioned items in the working copy that svn is afraid to overwrite.

The solution is to clean-up the working copy before doing the switch. By clean-up I don’t mean the SVN cleanup command, I mean deleting the unversioned and ignored files / directories. 

For those running Cygwin, the easiest way is to use svn status to feed the rm command:

svn status --no-ignore | grep '^[I?]' | sed 's/^[I?]//' | xargs rm -rf

For those who prefers native Windows solution, there’s a Powershell equivalent (from cultivating code blog):

(svn status "--no-ignore") -match '^[I?]' -replace '^.\s+','' | rm -fo -r]

For lazy folks like me, there’s a pure Tortoise SVN solution (this shows a list of files to be deleted, so you can’t use it in an unattended script):

tortoisproc.exe /command:delunversioned /path:"path/to/wc"

Finally, there is a manual way to do it with tortoise, as this StackOverlow answer explains: Hold down the shift key and right click on the working copy. There are now additional options under the TortoiseSVN menu including ‘Delete unversioned items…’.

In all cases, you probably want to keep the .suo files unless you want to collapse each of those 50 projects in your solution!

This post intends to fix the “file ‘/blog/post/2010/11/12/sp_findtext.aspx’ does not exist” error in my RSS feed.

Just a list of things I had at the top of my mind. There may be a few ones I missed and a few more obscure ones, but these should be the majority of the most important best practices.

• Maintainability and time-to-market
  • Modular architecture
    • MVVM
    • IoC/DI
    • S.O.L.I.D.
    • ORM for data access (NHibernate, Entity Framework)
    • SOA only for Ajax and for true business services (not for plain vanilla data access)
    • Command/Query separation aka PRG: post-redirect-get
    • Needless to say, keep most classes free from references to ASP.NET objects
  • Automated pro- and regression testing
    • Automated compilation and test execution on schedule or on check-in
    • Unit-tests (anything except MsTest, e.g. MbUnit)
    • Code coverage measurement. Code coverage never goes down policy.
    • UI tests (Selenium + RobotFramework)
  • Automated deployment
    • On-click deployment from the get-go. No manual error-prone steps.
    • WiX- or MSDeploy- based deployment
    • Automated post-deployment testing
    • Strive for transparent no-downtime deployment
      • Ensure that deployment does not interrupt users currently using the site
      • Precompile code to prevent compilation slowdown
  • Coding practices
    • No VB.NET, only C#
    • FxCop, NDepend to monitor coding standards
    • Most importantly: no huge pages, no long methods, no monster classes
  • Best of breed tools and technology for increased development productivity
    • ASP.NET MVC – no WebForms
    • ReSharper
    • LESS CSS
    • CSS Layout  frameworks
    • JQuery
    • YUI or other GUI component library
• Performance/Scalability
  • Defined performance requirements: throughput (reqs / sec), response time (page load time, time to complete basic operations), peek capacity (simultaneous users not necessarily clicking at the same time) etc.
  • Automated measurement of load and performance metrics (record Windows PerfMon counters and make available on a website)
  • Caching (at ORM level, at viewmodel level (memcached?), at page level (native IIS caching))
  • Enable gzip/deflate compression for both static and dynamic content
  • Async ASP.NET pages for long-running requests
  • Distributed in memory Session provider (no DB based)
  • Auto-minify javascripts & stylesheets
  • Make sure pages are compiled in release mode
  • Disable tracing
  • Avoid LINQ when working with large data sets
  • Use ORM profiling to ensure minimum interaction with RDBMS
  • Validate pages using YSlow and implement basic optimizations (javascript at the bottom, css in the HEAD etc.)
  • Implement load balancing using a web farm
  • Improve perceived performance by implementing a job queue system for long running operations (reports etc.)
  • Minimize use of ViewState
  • Validate controls on the client side
  • Use [Conditional] attribute for debug logging
  • Never return unlimited dataset
• Uptime & problem management
  • Defined SLA
    •  Required uptime window
    • Maintenance (designated downtime) window
  • Automated measurement of uptime metrics
    • % uptime against defined SLA
    • Mean time between failures
    • Time to recover operations
  • Problem detection
    • Automated health monitoring (aka pinging, live smoke-testing)
      • Using Selenium / Robot Framework
    • Error reporting
      • Use Elmah to report unexpected exceptions
  • Troubleshootability
    • Logging using log4net
    • Install Failed Request Tracing module and know how to use it
  • Failproof system
    • Analyze and avoid single point of failure
    • Implement failover clustering
•  Usability and user experience
  • Multi browser support
    • Test functionality in multiple browsers
    • Use reset style sheet
    • Use frameworks that isolate from browser differences (JQuery, CSS layout frameworks, component libraries)
  • Back button support and Refresh-button tolerance
    • Back-friendly URLs (using MVC route maps)
    • Command/query separation
  • Simplify complaints / feedback
    • http://getsatisfaction.com/
    • https://uservoice.com/
    • Twitter
  • Implement user-friendly error messages
• Security
  • HTTPS with real certificate (not self-signed)
  • License an automated software suite to test against SQL injection, XSS, XSRF, session hijacking, etc
  • Use an off the shelf authentication / authorization system, never design your own
  • Hire a hacker once a year

Just a bunch of links.

Vim tutorial for beginers

More advanced stuff

How to navigate around C source files in Vim and invoke the compilation from Vim

Even more tricks for programmers

Vim keyboard stickers

Vim screencast

Here’s what I’ve been up to lately: FreeBSD, vim, tmux, command line twitter client (ttytter) and unicode-capable graphic terminal (jfbterm). No X-Windows involved!

I have a table of Prices: ProductID, Date, Price and Source. My application needs to load new prices but if the price already exists and it is from a “better” source, it should update existing record.

The orignal app loaded all data in SQL temp table and did a bulk insert to the target table, followed by a bulk update. Kinda simple and effecient even if too SQL-centric. The first version of my app used to check every ProductID+Date combination for existance using regular stateful NHibernate session, then load new prices using SQL Bulk load, and update each existing price using regular NHibernate session. Needless to say, not only it was slow but the gap between the read and the insert allowed new prices from other sources (my app is not the only source) to sneak into the table causing duplicate/ambigous records to appear. I needed a solution that would allow me to Insert-or-Update records fast, without resorting to temptables+SQL mess. (I have a habit of trying to minimize the amount of SQL code in my application but if nothing else worked, I would have to fall back to the old ways.)

Here’s the outline of my new solution:

• Stateless NHibernate session (IStatelessSession) for extra efficiency. No need to spend CPU cycles keeping that first-level cache growing.
• Single explicit transaction wrapping entire load operation for all records. This reduces stress on SQL Server by not having it commit every record. If my volume becomes too high I can always limit the transaction size to say 10000 rows.
• Custom sql-insert script defined in the entity’s mapping file that implements the read-update-or-insert logic . The idea is to be able to completely process one record without going back-and-forth between SQL Server and the .NET
• NHibernate batching enabled, in order to push the records hundreds at a time to the SQL Server.

Combined, these techniques are designed to make the whole operation less chatty and, in a way, achieve the effect of having the processing done on the SQL Server side.

Here’s what I discovered while implementing the above:

• IStatelessSession does not support .BeginTransaction(IsolationLevel) overload. Instead, do session.Transaction.Begin(IsolationLevel.Serializable) which does exactly same thing (I looked at NH source code)
• You can further optimize the logic by not doing session.Get for each many-to-one-property of your entity. In my example, instead of doing price.Product = session.Get<Product>(ProductID) I can create a dummy Product object once and simply set its Id to a different value for each price. Stateless session does not care that the object is transient.
• The custom sql-insert script is sensitive to the order of parameters (they’re simply represented by “?”s). Turn on NH debug-logging and look at NH log to see the exact order in which object’s properties are mapped to SQL params. Do this before you implement sql-insert.
• The custom sql-insert script is sent to the server once for every inserted record. You may want to move it to an SP to minimize the traffic, and execute the SP from sql-insert. (Note that prepare-sql setting has no effect on batched statements)
• In order for NH batching to work, the entity’s Identity must be assigned before the insert. Can’t use generator=native (aka SQL Server IDENTITY).
• Since your Insert is now in fact Insert-Update-Or-Do-Nothing you want your entities Identity be its natural key, in my case ProductID + Date. That requires implementing Equals and GetHashCode but other than that it’s real easy.
• NH batching requires check=”rowcount” setting on the sql-insert. This means your script must return number of records affected, whether it actually inserted, updated or did nothing.
• NH batching uses global settings for its CommandTimeout, not the one defined in your SessionFactory. This means you must add hibernate-configuration section to your App.Config (or Web.Config).

This summary turned out to be more elaborate than I expected. No details for you then. You can figure it out, I have trust in you.

So a Java developer, a web developer and a UNIX admin go to a bar to get a drink. The Java developer and the web developer get into an argument over who’s job is more important. “Thousands of people see my work” claimed the trendy web developer. “My code runs the banks!” retorted the tidy Java developer. They both look to the UNIX admin, expecting him to have the final word. He puts down his drink slowly. He then smashes his glass on the table, jumps on it and rips off his pants. He takes a shit and rubs it in the Java devs face, and while the web dev makes a run he jumps him, ripping his eyes out and eating them. He then goes back to the cave where he lives.

In order to make bash shortcuts like Alt+. work in virtual terminals, you need to modify the keyboard mappings to map the left Alt to work like Unix special “Meta” key.

You can either start with one of the existing mappings, like this:

sed 's/lalt/meta/g' /usr/share/syscons/keymaps/us.iso.kbd >  /usr/share/syscons/keymaps/local.kbd

Or, if you’re not sure which keymap file you’re currently using you can get your current keybindings using kbdcontrol:

kbdcontrol -d | sed 's/lalt/meta/g' > /usr/share/syscons/keymaps/local.kbd

Then, assuming you want this change to be global, edit your /etc/rc.conf file and add the following line:

keymap="local"