One of them was that I was working on a team where a programmer quit and I had to get a very complex codebase ready for production. The last programmer was terrible, the kind of guy who had trouble making primary keys that were unique, where any code that could possibly have a race condition did, and so forth. The code had unit tests, however, and that made it salvagable, and eventually I got the system to a place where it worked correctly and the customers loved it.

In nine months of effort on this, I ran into one refactoring where it felt the unit tests were a burden, and that involved about a day of work rewriting the tests. Unit tests are more likely to be a problem, however, when they add to the time of the build process. For instance, I wrote something in JUnit that hammered part of the system for race conditions, and this was key to fixing races in that part of the system. It fired off a thousand threads and took two minutes to run, and adding two minutes to your build is a BIG PROBLEM, particularly if anybody who wants to add two minutes to your build can do so and if anybody who wants to remove two minutes from the build is called "a complainer" and "not a team player". Overall the CPU time it takes to run is more likely to be a problem than the developer time it takes to maintain them.

As for Mockito I have found it is a great help for writing Map/Reduce jobs. As I don't own a big cluster and as I sometimes like to code on the run with my laptop, an integration test typically takes ten minutes with Amazon Elastic Map/Reduce. It takes some time to code up tests, but I get it all back with dividends because often I get jobs running with two or three integration test cycles instead of ten or twenty. When I find problems in the integration tests, usually I can reproduce them in the unit tests and solve them there.

Now, it did take considerable investment to get to the point where unit testing worked so well for me. I used to have problems where "the tests worked" but the real application didn't because Hadoop reuses Writable objects so if you just pass a List of objects to the reducer, you might get different results in a test than you do in real life. Creating an Iterable object that behaves more like Hadoop does solved that problem.

Generally if you are feeling that "unit testing sucks" or "mockito sucks" it's often that case that you're not doing it the right way.

Well explain further. I hate these smart arse sounding comments - "you are doing it wrong" without any indication why, or how to do it better.

Thus any test that is written in such a way that it would be present an issue in refactoring code should be avoided if at all possible. A simple example is directly constructing the class under test in the test method:

  @Test 
  public void tryToAvoidDoingThis() {
     MyClass = new MyClass(param1, param2);
     // do stuff to my class
   }

Instead, have a level of indirection and have a single method that can create a sample MyClass. Now when the parameters change, only one construction site has to be updated.

In general, unit tests should not be testing specific / internal implementation details of the class. Rather, the tests should verify the documented public behavior of the class.

Interestingly it's also the crux of good design. Breaking code up into composeable pieces that aren't braided together.

Other examples may be around a `setup` method. If the method is private, don't test it. Then you can refactor freely. If it's public, test the pre/post conditions around the method in as few places as possible (hopefully one). Even if other tests rely on the object having been "setup", just trust that it works. If the specification of `setup` changes, you just have the `setup` tests to update, not the entire object.

Second, start writing tests to verify bug reports and then fix the bug. In large systems I find this critical to honing in on the exact problem. The mental exercise of crafting the test to trigger the bug helps me really understand the problem.

Finally, start new features by first writing a test to simply drive your new codes golden path. When working in a large system I find writing a test to run my new code a much faster development turn around time than rebooting the entire system. This is compounded when there are many systems or moving parts which is common today.

With unit tests, there are certain things that must be tested, such as very high-value code contracts, and the like. There are many things that people test (like "correct output" for the input) which may not be so valuable, particularly if several possible values may be correct.

So test contracts, not internals, and not representation. And please for the heaven's sake, don't test the behavior of your dependencies.

Unfortunately I also believe those comments are generally true, and I also believe when the posters answer "why", they will give you an answer that is also doing it the wrong way.

I have no idea how to do unit testing. I only believe there is a right way.

This doesn't make sense, how could you trust his unit tests?

For better or worse, this system had a number of data-centric objects; these objects passed many correct tests, but they failed to be deserializable from XML (because of the way collections were handled) as well as having other deficiencies.

The tests meant I could fix those deficiencies quickly and have faith that I fixed them correctly. Of course, I added new tests to test that the system did the things it had to do.

I write unit tests for every one of my core algorithms. I have also seen a massive amount of dumb tests written by TDD people who strive for 100% code coverage. Ridiculous. It takes as much time to maintain the (often useless) tests as it does the code. Where they make sense they're indispensable.

I think the fundamental question is why 100% code coverage is important. The fact is that it isn't. the problem with TDD is that a lot of people who do it totally get the idea of why you are testing entirely wrong. The goal should never be to "ensure your code functions properly." The coal should be to "ensure the code contracts are adhered to." If you test with that in mind, you will write lots of unit tests and almost never have to rewrite or delete them due to fixing bugs.

Again this comes back to what I said in another comment that tests should never be written to the code. Once you get that, then code coverage ends up being meaningless and not something you want to worry about.

Big projects over some size normally do it that way.

More specifically, consider an SSE2-based function 'float32 floor(float32)'. There's only about 4 billion inputs, so why not test them all? That only takes a minute or so.

How is testing 100 inputs a unit test and testing 4 billion inputs, through exactly the same API, an integration test?

As the author points out, many people wrote libraries which are supposed to handle the entire range, but ended up making errors under various conditions, and even given wrong answers for over 20% of the possible input range.

The statement was very causal.

> If you've got a slow test suite, that mean's you've got a bad test suite.

Or you have a fantastic test suite that also includes integration tests. See?

If you say it's slow then either it's a bad test suite, and/or it includes integration tests. I believe that is the logic, yes?

There is no lower unit to test, so therefore this must be a unit test.

The linked-to page shows that testing all possibilities identifies flaws that normal manual test construction did not find. Therefore, it must be a better test suite than using manually selected test cases, with several examples of poorly tested implementations.

(Note: writing an exact test against the equivalent libc results is easier to write than selecting test cases manually, and it's easier for someone else to verify that the code is testing all possibilities than to verify that a selected set of corner cases is complete.)

Therefore, logic says that it is not a bad test suite.

Since it contains unit tests and it is not a bad test suite, therefore it must not be slow.

Therefore, 4.5 minutes to unit test these three functions is not "slow".

Therefore, acceptable unit tests may take several minutes to run.

That is what the logic says. Do you agree? If not, where is the flaw in my logic?

A good test suite has two qualities - how comprehensive it is and how fast it takes to run. If either is lacking, then it is no longer a good test suite.

We had a series of tests (more towards integration tests I guess) at one point in LedgerSMB that did things like check database permissions for some semblance of sanity. These took about 10 min to run on a decent system. The reason was we stuck with functionality we could guarantee wouldn't change (information schema) which did not perform well in this case. Eventually we got tired of this and rewrote the tests against the system tables cutting it down to something quite manageable.

We had this test mixed in with db logic unit tests because it provided more information we could use to track other failures of tests (i.e. "the database is sanely set up" is a prerequisite for db unit tests).

Either that, or the person just hasn't been sufficiently burned by someone changing something you're not aware of and having to track down a run time error for days that could have been caught and fixed by a unit test in minutes.

I read the article, and much of what he speaks of is tautological unit tests - testing something where someone could never have done anything but that. I've seen people unit test the behavior of filling up a collection then test to make sure the collection has all of those elements, for instance. And there, he has a point.

But there's a dangerous line there. And while the article makes a good point that too much of that type of testing can be detrimental, I've generally found it better to err on the safer side of more tests.

On your own project, or even on a small team, you can probably get away without them much of the time. But, on larger projects, where many different developers sometimes go back and make changes to code they didn't write, it's very easy for a new guy/gal to make changes with unanticipated consequences. When that occurs without sufficient test coverage, the project will wind up spending 10-20x more man hours to fix the issue.

Each test is extremely fast. Testing all 12+ billion cases takes 4.5 minutes. Are these unit tests or integration tests?

If they are integration tests, roughly where is the boundary between the two?

Is it meaningful to distinguish between "unit" and "integration" testing based only on the amount of time they takes? That is, if the unit tests take 0.1 second too long then do they suddenly become integration tests?

I think the problem is that definitions that should reflect the granularity of what is being tested have become too interconnected with assumptions about frequency of testing / time to run and it has created completely mangled definitions.

The problem is so bad that some developers I've crossed have the firm opinion that mstest should only be used for "unit" tests. It is frustrating.

If you have slow unit tests, don't run them on every compile. If you have fast integration tests, run them as much as you'd like. I've personally never defined unit tests as "things that run fast", despite that being a valuable property it does not seem essential to the definition, but perhaps I have the wrong understanding of what a unit test is.

The longer your unit tests take, the less likely people will be to use and run them often, which is the whole point. Let the nightly build on the CI machine exercise the long running tests when everyone is asleep.

Really, I think the problem is that unit test frameworks are currently incapable of doing the right testing.

Unit tests take quadratic time. That is, each new test requires running all previous tests, to get the green. And at some point, a project will have enough tests that it can't finish in 10-15s.

One option is to mark "fast" and "slow" tests. Another is to recategorize them as "unit" vs. "functional" tests.

These are poorly-defined labels. In this case, the 4.5 minutes of testing is "slow", yes, but it only needs to be run when a specific, small part of the code changes. The problem is, there's no way to determine that automatically. The test runner can't look at the previous test execution path and see that nothing has changed, and there's no way to mark that a test should only be run if code in functions X, Y, or Z of module ABC has changed.

Humans are able to figure this out. Well, sometimes. And with lots of mistakes. Get the unit test framework to talk with a coverage analysis tool, plus some static analysis and perhaps a few annotations, and this discussion of how to distinguish one set of tests from another disappears.

Blue-sky dreams. I know. :)

In real life we toss those functions into their own library, note that the code is static, and do the full test suite only occasionally; mostly when the compiler changes.

In other words, bypass CI the same way one does any other third party library. (How often do you run the gcc test suite?)

That still means to run all the tests each time, with re-prioritization to enrich the likelihood of faster feedback.

But if none of the code paths used for a test have changed, and the compiler hasn't changed, and there's nothing which depends on random input or timing effects, then why run those tests at all?

The reason is we don't have a good way to do that dependency analysis, which is why we run all of the tests all of the time. Or we manually partition them into "slow" and "fast" tests.

So now we have to write tests for all our code, and tests that run fast. We have to refactor our code around the tests. Something seems a bit back to front here. Or is the assumption that if we have 100% test coverage that runs fast then it means that we have written the best code possible?

I think I am siding with the author of the article on this one.

The author complains about re-factoring code into smaller testable functions. I completely disagree. Code structured as small easily understood functions which do one thing and have obvious inputs and outputs is good code which is much easier to extend and modify.

Sounds to me like the kind of tests you're describing aren't necessarily unit tests (system-level race conditions aren't typically discoverable with a simple unit test); where the tests were truly unit-level, the proposed alternative (assertions) may have been even more informative. Finally - a few real unit tests for known correct behavior are advised where in essence the algorithm can be described independently from its outcome.

I think the only verification that should be done with mocks (the verify method) is that important collaborators have been called. Things that run off and mutate some global state, or perform an action.

Stubs are useful for controlling the execution state of the method under test. You could stub DateTime such that it was midnight in one test, and noon in another, if the behavior of the function relied on that. It does introduce a dependency that has to be changed along with the code, but is often helpful.

I do agree with the author that there's an art to testing appropriately. You could mechanically write testing code for every line of code under test (if statement with single condition? Two tests to verify each path is taken!), which leads to your app being coded twice: once in code and once (or more) in the tests.

And that's the rub. Mocks are useful. They help you write tests that are more robust, without having to resort to massive and complex @Befores. At the same time, though, they have an effect that is more negative than positive on the ability to change the code under test.

Say I have three layers of custom code: A calls B, which calls C.

If I want to test B, then I want to mock C, and have my test call B similar to how A does. I want to mock C because C is also custom, and if my test fails, I want to know if it's because of bad B implementation, and not because a buggy C might be confusing matters.

But if B also calls D from an external open source or vendor-supplied library, I don't usually want to mock D. That just adds needless complexity to the test, and reduces my focus on my own custom code.

An exception would be if this library code makes its own network call or something - then you might want to mock it to save time.

Anyway, mocked unit tests become far simpler if you maintain the right focus. Use test A to call B (passing in canned fixtures if necessary), while B mocks C only to maintain focus on B's implementation. If you start getting involved in trying to mock external library code, or even internal private methods that B calls, you'll have a bad time.

The advantage to maintaining that kind of focus is that refactoring becomes easier. Want to change the name of C? Your IDE should handle refactoring your test, too. Want to change the implementation of B? You don't even need to change your test at all, just make sure the right values are still there in the return value. Maybe you'd need to add a couple of assertions, but that's it. If you're looking at having to do a serious refactoring of your unit test in those cases, then it probably just means you're still designing your code architecture and things are still really fluid. And in that case, it would make sense that you might have to throw away your test, because by definition it means you are still deciding on what your specifications are.

I think that building that level of isolation between you and your framework or library is just basic good practice. The fact that you need time doesn't change, but the way that you get it might.

I'd rather have a program that does what it does in X lines of code, than a unit tested, mocked, codebase in 5X lines of code. Sure you have tests, for whatever they're worth (I'm somewhat skeptical of TDD in the first place), but you have so much MORE code.

I'm similarly wary of convenience libraries that provide marginally simpler APIs on top of standard libraries.

I'm not convinced it's a good idea. I wish I had your experience. Any good reading material?

Anything on the Adapter pattern.

Wrapping a library in your own concept allows you to define what's right for your application. It has the effect of pushing the third party library out to the edges of your system, replaced with whatever you wrapped it with. This makes replacing it, for testing or any other reason, much easier than if it's proliferated throughout your code unwrapped.

Wrappers should be simple so creating them and documentation, beyond a few integration tests to understand how the library works, shouldn't be a huge concern.

I've been doing a lot of Javascript stuff lately and libraries like Dojo have 5 different ways to locate an element in the DOM. I have no idea if all these ways will be around in a future release, or, right now, which one is better. Unifying the Dom select code behind our own interface keeps things uniform throughout the app, instead of each of us using a different function, and lets us try out the different library functions, or different libraries, easily.

"Working Effectively With Legacy Code", by your esteemed parent poster... :)

For example (Perl example here), in LedgerSMB we layer some things. We layer the templating engine. We layer (now, for 1.4) arbitrary precision floats. We layer datetime objects. Many of these are layered in such a way that they are transparent to most of the code.

But there are a lot of things that aren't layered because there isn't a clear case for so doing right now.

(As a footnote, PGObject requires that applications layer the underlying framework because there are certain decisions we don't feel comfortable making for the developer, such as database connection management.)

Low density code that doesn't provide application logic is one of my pet peeves.

There is a philosophy that started in the 90s (and Microsoft was proponent of it [1]) that adding more layers to an application would make it more malleable, but in a typical CRUD web app, layers only bloat the code and make it slower.

I'd suspect adding a wrapper to a date class just for the sake of testing is more likely to add bugs than remove them.

[1] http://www.asp.net/web-forms/tutorials/data-access/introduct...

Write adapters or facades that wrap external libraries, use those in your own code's unit tests. This makes you less bound to a specific library as well. Don't mock the outside world [0] for testing your adapters/facades/whatever, but do integration tests that cover your adapters using the real outside world.

You'd also do complete end to end tests where the entire system is used as if it were in production (acceptance testing). TDD makes a lot more sense if you think of it in those three layers: acceptance, integration, unit.

[0] Outside world means anything that's not your code -- networking, filesystem, external libraries, etc.

This is not a bad thing. Ostensibly, you're making a change to code which implements a business function for a reason. If you're not using DateTime anymore, there should be a reason why.

Tests are supposed to be the proof that a business function is performed correctly.

If date/time is such a fundamental piece of logic that you have many tests that verify various conditions based on what day or time it is, and you change the fundamental WAY you represent date and time, then why is it bad that you have to make a lot of test changes as well as code changes?

Having those tests, and changing them over should be worth the effort to give you the confidence that your new implementation of how a fundamental of your code works the same way as before.

As far as the concept of checking "this method on this collaborator will be called X times," that can be painful to get correct the first time, but say you again change business logic and add a new condition. Is it not worth the effort to either think about "Oh, now I will have to call getHour one more time" and fix the test, or discover a broken test AND CONSIDER WHY you're being told "expected 2 calls but got 3" ?

Granted, if you're just changing times(2) to times(3) and going on, well, then don't bother. But that SHOULD be telling you something valuable. It's up to individual developers whether they choose to see that value or not.

Some unit test styles enforce a given API at all levels in the code. Others restrict themselves to tests through certain restricted APIs.

For example, the SQLite code tests through the API, so it's possible to do major revisions of the code without changing the tests. The choice of how date logic works is invisible to the tests. Date tests can be done through SQL, without testing the actual code functionally equivalent to DateTime.

I prefer the latter. I liken it to building a bridge. The main test is "can you safely get traffic from X to Y", with other tests related to maintenance, beauty, environmental impact, and so on. This can start with a stone bridge, then a truss bridge, or a suspension bridge.

It's also possible to have tests like "there must be a pillar here, a truss there, an arch up there, and a screw over there", which requires that the bridge must be an iron bridge using a Pratt truss design, with three support columns made of granite.

Rebuilding/refactoring to even, say, a Parker truss would require rewriting some of the tests. Rebuilding/refactoring to, say, a truss arch requires a lot of rewritten tests.

But under the high-level tests, like for SQLite, major redesigns don't require major changes in the tests.

If at the start you know that you want a specific bridge, and you just have to work on it, then go ahead and write these very specific tests. Just remember that major refactors will require a lot of rewritten tests.

In my opinion the value in testing is verifying what an interface should do, not how it does it.

    var thing = new Thing();
    thing.shouldBe.instanceOf(Thing);

1. It can save time debugging failures because I know whether the basic instantiation of thing was sane or not.

2. There are cases where there are complex object instantiation tests where an object is returned according to some other logic. Obviously these need to be tested.

In the first case those aren't tests for the sake of testing the API. They are sanity checks for saving time troubleshooting other test case failures.

my $thing = $thingloader->new($some_complex_data); ok ($thing->isa('Specific::Kind::Of::Thing')

is fine and useful.

At the very most you should be expecting an interface, and not having the correct one should yield a runtime error within the code to begin with.

The details of how MyClass calculates due dates should be irrelevant. The test in your example violates this. It makes privileged assumptions to assert implementation details.

This is why it's better to code and test against interfaces. This makes it harder to make any assumptions about the implementation because it puts the abstraction front and center. A test suite written against an interface can be applied to multiple implementations. This of course requires some dependency orchestration to load different implementations into the suite, as well as to create/avail any needed mocks for that implementation (which ideally are also declared as interfaces). But this limits ripples from implementation changes to the dependency orchestration code.

With that in mind, if an implementation's internals need testing, then tests can be written against the class itself (rather than any interface). If the class has dependencies, then it should be obvious (or documented) how/when those dependencies will be accessed. When necessitated by the class's outward responsibilities, this makes it possible to ensure that a dependency is correctly accessed in certain conditions. E.g. Perhaps it must be ensured that an order processor always asks for validation from a credit card gateway. Your example does not meet this standard.

Basic theory aside, I personally agree that high coverage is not worthwhile. I prefer tests that ensure bug fixes don't regress, or that are high-level and mimic end-consumer behavior.

I generally prefer to write a mix of unit and integration tests, and I typically find the integration tests to be more useful.

Sometimes it can be useful to make this kind of assertions on a mock though. Say you're mocking a web service, you can use this kind of assertion to ensure you're calling the web service properly. In this case it's useful because it's verifying behavior at a boundary between your program and the outside world. That's what I think tests are for. The outside world is not made only of users. Communication with external resources is also important and can be verified with mocks. In other words, this kind of mock assertions is interesting to verify the indirect outputs of the program: http://xunitpatterns.com/indirect%20output.html

Verification testing does however have it's uses in terms of verifying that key interactions have happened in the way you expect.

In the most part it's used correctly, assertions are much more widely used than verification testing.

On the other hand, I've also seen it used overly invasively -- that is, the test writer uses it to verify the status of some object internal to the class under test, like your example above. As you note, it creates an incredibly tight coupling between the test and the method being tested.

In general, I think that this sort of thing is correlated with classes that are highly stateful. If, for example, your class is full of void methods that modify class members, then you may have no other option than inspecting the internals of the class in order to test its functionality. OTOH, if you can manage to write things in a more functional, less state-heavy manner, then you need less of this sort of introspection.

If you don't mock out classes around the class your testing, its a integration test.

This is exactly my sentiment. I would go even further, though, and say that "the more powerful the mocking framework, the more it discourages design improvements." If you have 50 or 60 references in your test to a mocked class of type A, and A turns out to no longer be needed, then the effort to get rid of it is higher than it otherwise would be.

Note, I am very much for unit tests. However, I recognize that the basic test cases really are software engineering rather than development territory. So I think this whole argument points at a much deeper problem, and one I don't think test driven development is the answer for.

let's draw a parallel to the Python midnight time as boolean discussion for a moment. The Python midnight time issue was entirely because the developer of the library made his code the contract. "This is how it works so hence live with what that means" is not really a contract and in that case it had some really nasty corner cases (timezones west of GMT never evaluate to false for any time with a timezone, but timezones UTC and east evaluate to false when the time represents UTC midnight). But a lot of people didn't want to break that contract. Really? That's a contract that could only be appreciated by tax lawyers.

The same fundamental problem happens with unit tests. "This code works so it must be correct" is a typical response and so the code becomes the contract. A bug is found, and fixed, and the test cases start failing. This is a waste of time. It is a waste of time to write the test. It is a waste of time to remove it because it is irrelevant.

My recommendation is to do extensive unit testing. However, take the unit tests seriously. Before you write a test, ask yourself "If a bug is found and fixed in the software or any of its dependencies, should this test ever fail?" If the answer is "yes" then don't write the test.

The point of unit tests are to ensure that software contracts are not mistakenly broken. This is true no matter how you write code. My test scripts are always longer code-wise than my production code. This is true for SQL and PL/PGSQL, just as it is for Perl. However, I try hard to ensure that a test failure means that a previously working API is no longer working rather than that I changed something.....

But, as for TDD, I think people can go way, way overboard. There are serious diminishing returns to doing TDD 'the right way'. I'm quite satisfied using unit tests to test particularly complex business logic / domain logic and as integration tests. I find that all too often devs will get so enamored with their 100s of tests that basically test nothing (as a ratio of functionality to lines of code), that they'll forget the main goal: shipping relatively high-quality software.

Disclaimer: I started as a hack, worked for many years to become a non-hack, then devolved to a hack with caveats. Now, for me, it's all about achieving the right balance between speed and perfection. YMMV.

Edit: shipping [relatively high-quality] software

It's not.

Software is a means to an end. A tool we built for users and customers to solve their problems.

Software that doesn't solve the customer's problems might as well be gravel on a beach.

Software that is shipped, but isn't maintained, will diverge from customer's needs as they change. And will become just as useless.

So are you trying to precisely understand a customer's key requirements? Then you best represent them in unit tests.

Are you building and releasing iteratively and getting customer feedback to make sure that customers are getting what they actually need? (A lot of times what people say they need changes once they actually get what they ask for) Then you need to be able to refactor frequently and fearlessly, and you best have tests.

Are you considering future releases after shipping major versions? Same, you will need to refactor, and you will break existing functionality, and you need tests.

I understand you want to be pragmatic. But don't think that the people that preach TDD are just sitting there in love with their best practices for their own sake. We have just been burned too many times and believe the best process and framework for delivering customer value via software is to build on a solid foundation.

On second thought, I think it's an interesting topic of conversation.

When I'm working, my primary goal is to make money. And, when I'm working for a client or boss or customer, that means being responsive and showing them value for _their_ money. Oftentimes, this means trying to protect them from themselves, but, again, I want to get paid. So, eventually, despite all my arguments, it turns back to me shipping software. They want to 'see' something for my expensive invoice, subscription fee, or salary. The payer doesn't care about all that 'stuff' that I do that they can't 'see'.

So, yes, you're right. I do sacrifice quality at times to ensure that I get paid. It's not intentional. No way. But, I am a biased participant in this debacle: I have a perverse incentive to ship non-perfect software. I don't intentionally try to do so, obviously. But, there is an equilibrium point where I stop arguing and see that I can fix it in v2.

Now, on my personal projects, I actually take my time. I do things the right way, because I have time on my side. I'm my own boss. And, in those cases, I actually do write full sets of unit tests.

Interesting for me to observe this in myself. Thanks.

But, yes, when I'm working for someone, I have considerable pressure to break the rules. You're right - it is a trap.

I like tests. When a section of code is complex, and can't reasonably be refactored, a test is a good way to help build confidence that you've driven the wonkiness out of it, and don't reintroduce it with further changes. They help you catch dumb mistakes, where you change how you're passing a reference, or the type of child object a function sees, and suddenly, you're getting subtly wrong output. I do a lot of Computational Fluid Mechanics (CFD), and we've got a standard suite we run when we compile to make sure new changes haven't broken our base functionality.

Still, in most cases, I'd much rather look for a refactoring or functionality combination option. I'd rather each piece be small enough that I can hold a mental model of it in my head, and not be too scared I'm missing something. However, also still a bit of a hack.

I use mocks more-or-less as advanced stubs, because when something has 600 methods in it (legacy code base fun) and I need one of them I don't particularly feel like generating that entire stub.

This is more of a problem where a library is not used well. Mockito is a very good library, this example really does it disservice.

The above test is much easier without mocks. You construct a date, pass it in, then check that the result is equal to the input plus 10 days. That's all you care about, so that's all you test.

Verify is useful if you have to update external state. If, in your contrived example, for whatever reason, you needed to keep a tally of all of the times that due dates were ever calculated and that was stored somewhere else, you might have a test that mocks your CalculationCounterClass and verifies that a method (addCalculatedDueDate or whatever) gets called on there...

Good, simple (and IMO brittle) tests that confirm what you care about are useful. Forcing a particular implementation through your tests is a drain.

I'm not a TDD stickler or anything, but tests are what make refactors even possible. Will tests catch all the issues? No, but they will make sure the part you refactored still interacts properly through its public interface.

If some tests have to be changed that is even better because those tests hopefully contain corner cases that have been caught and tested for over the life of the original code, and will force the person doing the refactor to think about those issues.

Perhaps in general tests provide a useful safety net for refactoring, but I think the benefits are often overstated. In any case those tests don't have to be unit tests.

If I'm working on the kind of system that would need mocks for comprehensive unit testing, I'd rather either use integration or high level functional tests if that kind of strategy is viable. At least then I'm still testing that my actual production code works.

If that's not possible, for example because I'm integrating with an external system where I can't test operations that would cause real side effects, I'd rather look for a different strategy entirely. For example, I might change the design to better isolate the bits of code I can control and test myself, I might (re)implement riskier areas in languages with powerful systems that will guarantee various classes of programmer error can't happen, or I might not have an automated test suite covering some code at all and rely on things like code review or automated code analysis tools instead.

Well, there's your problem right there. Do not implement a crap workaround when you know what the proper solution is.

That just looks like a horrible test, I tend to argue that when you have to resort to using a verify on any mock then you are looking at something problematic.

There are ways to do it and ways not to do it, though. A good framework should be exhaustively unit tested. Individual components that use that framework should be lightly unit-tested (rather than having the framework heavily mocked) and more attention should be spent on integration/acceptance tests.

Finding bugs is not the primary purpose of unit tests.

>not for exposing nullpointer exceptions

If you are using a language primitive enough to have null pointer exceptions, then using unit tests to try to detect some of them is a perfectly reasonable thing.