There is such thing as too much testing
Lessons Learned
It is supreme importance to keep your test suite fast and reliable
Unreliable test suites
Devs stop wondering what caused the fail and 'fix' the build in bad ways
Team will stop caring about fails
Slow suites
Fix it
Remove redundant tests
Profile your suite and examine slow tests
Stop hitting external services or database
Run on a fast machine or in the cloud
Parallelize
Start to see the test as an obstacle
Spend a lot of time on twitter
Devs won't check in near end of day
Integration tests can replace unit tests if you're not careful
Selenium/watir tests are deceptively easy to get into but hard to maintain
Start small if the team is new to testing
What's the Right Level of Testing
From the application's point of view
What level of assurance does it need?
Someone have to fill the form second time
Someone looses life
Somebody looses money
From the team's point of view
What's the level of buy-in for managers
What's the level of developer buy-in for testing
gridlock, poor velocity
morale problems
From the code's point of view
Tests (integration tests especially) should be catching bugs
Testing is supposed to be about good design
The Tour
The Social Behemoth Part III
Above 90% coverage
SQuiD
Right for the project - had to be out of the door really quickly!
No integration testing
No view testing
The Social Behemoth Part II
60% coverage
The VOIP Signup App
ridicilously well-tested code
RoR
very high quality demanded
The Social Behemoth
0 - 50% coverage
only 50% of the team (consultants) wrote tests
they saw tests as impediment
us versus them philosophy
even managers could write some code
if you write a spike do not start building on top of that
The Small Social Start-up
Extensive Selenium testing
not so good if your doing it just to test if it works in the userspace
bulletproof app
The IP Space Manager
Government
Ruby/Rails project
The Huge Financing App
test suite
it probably made poeple in the company feel better
they didn't know why they did run them
80% of these tests failed!
8 computers 36 hours
test hostile world
EJBs 1.0 shit
21% coverage, 25% 4 years later
The Dealer Portal
WebSphere
a bit heavy
webunit tests
slow build (15-20 minutes)
a lot of them
small project (6 devs/5 months)
