How should I unit test threaded code?

I have actually so far stayed clear of the problem that is testing multi - threaded code given that it simply feels like way too much of a minefield. I would certainly such as to ask just how individuals have dealt with testing code that relies upon strings for effective implementation, or simply exactly how individuals have dealt with testing those sort of concerns that just turn up when 2 strings connect in an offered fashion?

This feels like an actually key trouble for designers today, it would certainly serve to merge our expertise on this set imho.

770
2019-12-02 01:18:23
Source Share
Answers: 8

Another means to (kinda) examination threaded code, and also really intricate systems as a whole is via Fuzz Testing. It is not wonderful, and also it will not find every little thing, yet its most likely to be valuable and also its straightforward to do.

Quote:

Fuzz screening or fuzzing is a software program screening strategy that gives arbitrary information ("fuzz") to the inputs of a program. If the program falls short (as an example, by collapsing, or by falling short constructed - in code assertions), the issues can be kept in mind. The wonderful benefit of fuzz screening is that the examination layout is exceptionally straightforward, and also devoid of assumptions concerning system actions.

Fuzz screening is usually made use of in huge software program growth tasks that use black box screening. These tasks generally have a spending plan to create examination devices, and also fuzz screening is just one of the strategies which supplies a high advantage to set you back proportion.

Nonetheless, fuzz screening is not an alternative to extensive screening or official approaches: it can just give an arbitrary example of the system is actions, and also in a lot of cases passing a fuzz examination might just show that an item of software program takes care of exemptions without collapsing, as opposed to acting appropriately. Hence, fuzz screening can just be considered as a bug - searching for device as opposed to a guarantee of top quality.

15
2022-07-16 15:32:43
Source

I've done a great deal of this, and also of course it draws.

Some tips:

  • GroboUtils for running numerous examination strings
  • alphaWorks ConTest to tool courses to create interleavings to range models
  • Create a throwable area and also examine it in tearDown (see Listing 1). If you capture a negative exemption in an additional string, simply assign it to throwable
  • I developed the utils class in Listing 2 and also have actually located it very useful, specifically waitForVerify and also waitForCondition, which will substantially increase the efficiency of your examinations
  • Make excellent usage of AtomicBoolean in your examinations. It is string secure, and also you'll usually require a last reference type to store values from callback courses and also suchlike. See instance in Listing 3
  • Make certain to constantly offer your examination a timeout (as an example, @Test(timeout=60*1000)), as concurrency examinations can occasionally hang for life when they are busted

Listing 1:

@After
public void tearDown() {
    if ( throwable != null )
        throw throwable;
}

Listing 2:

import static org.junit.Assert.fail;
import java.io.File;
import java.lang.reflect.InvocationHandler;
import java.lang.reflect.Proxy;
import java.util.Random;
import org.apache.commons.collections.Closure;
import org.apache.commons.collections.Predicate;
import org.apache.commons.lang.time.StopWatch;
import org.easymock.EasyMock;
import org.easymock.classextension.internal.ClassExtensionHelper;
import static org.easymock.classextension.EasyMock.*;

import ca.digitalrapids.io.DRFileUtils;

/**
 * Various utilities for testing
 */
public abstract class DRTestUtils
{
    static private Random random = new Random();

/** Calls {@link #waitForCondition(Integer, Integer, Predicate, String)} with
 * default max wait and check period values.
 */
static public void waitForCondition(Predicate predicate, String errorMessage) 
    throws Throwable
{
    waitForCondition(null, null, predicate, errorMessage);
}

/** Blocks until a condition is true, throwing an {@link AssertionError} if
 * it does not become true during a given max time.
 * @param maxWait_ms max time to wait for true condition. Optional; defaults
 * to 30 * 1000 ms (30 seconds).
 * @param checkPeriod_ms period at which to try the condition. Optional; defaults
 * to 100 ms.
 * @param predicate the condition
 * @param errorMessage message use in the {@link AssertionError}
 * @throws Throwable on {@link AssertionError} or any other exception/error
 */
static public void waitForCondition(Integer maxWait_ms, Integer checkPeriod_ms, 
    Predicate predicate, String errorMessage) throws Throwable 
{
    waitForCondition(maxWait_ms, checkPeriod_ms, predicate, new Closure() {
        public void execute(Object errorMessage)
        {
            fail((String)errorMessage);
        }
    }, errorMessage);
}

/** Blocks until a condition is true, running a closure if
 * it does not become true during a given max time.
 * @param maxWait_ms max time to wait for true condition. Optional; defaults
 * to 30 * 1000 ms (30 seconds).
 * @param checkPeriod_ms period at which to try the condition. Optional; defaults
 * to 100 ms.
 * @param predicate the condition
 * @param closure closure to run
 * @param argument argument for closure
 * @throws Throwable on {@link AssertionError} or any other exception/error
 */
static public void waitForCondition(Integer maxWait_ms, Integer checkPeriod_ms, 
    Predicate predicate, Closure closure, Object argument) throws Throwable 
{
    if ( maxWait_ms == null )
        maxWait_ms = 30 * 1000;
    if ( checkPeriod_ms == null )
        checkPeriod_ms = 100;
    StopWatch stopWatch = new StopWatch();
    stopWatch.start();
    while ( !predicate.evaluate(null) ) {
        Thread.sleep(checkPeriod_ms);
        if ( stopWatch.getTime() > maxWait_ms ) {
            closure.execute(argument);
        }
    }
}

/** Calls {@link #waitForVerify(Integer, Object)} with <code>null</code>
 * for {@code maxWait_ms}
 */
static public void waitForVerify(Object easyMockProxy)
    throws Throwable
{
    waitForVerify(null, easyMockProxy);
}

/** Repeatedly calls {@link EasyMock#verify(Object[])} until it succeeds, or a
 * max wait time has elapsed.
 * @param maxWait_ms Max wait time. <code>null</code> defaults to 30s.
 * @param easyMockProxy Proxy to call verify on
 * @throws Throwable
 */
static public void waitForVerify(Integer maxWait_ms, Object easyMockProxy)
    throws Throwable
{
    if ( maxWait_ms == null )
        maxWait_ms = 30 * 1000;
    StopWatch stopWatch = new StopWatch();
    stopWatch.start();
    for(;;) {
        try
        {
            verify(easyMockProxy);
            break;
        }
        catch (AssertionError e)
        {
            if ( stopWatch.getTime() > maxWait_ms )
                throw e;
            Thread.sleep(100);
        }
    }
}

/** Returns a path to a directory in the temp dir with the name of the given
 * class. This is useful for temporary test files.
 * @param aClass test class for which to create dir
 * @return the path
 */
static public String getTestDirPathForTestClass(Object object) 
{

    String filename = object instanceof Class ? 
        ((Class)object).getName() :
        object.getClass().getName();
    return DRFileUtils.getTempDir() + File.separator + 
        filename;
}

static public byte[] createRandomByteArray(int bytesLength)
{
    byte[] sourceBytes = new byte[bytesLength];
    random.nextBytes(sourceBytes);
    return sourceBytes;
}

/** Returns <code>true</code> if the given object is an EasyMock mock object 
 */
static public boolean isEasyMockMock(Object object) {
    try {
        InvocationHandler invocationHandler = Proxy
                .getInvocationHandler(object);
        return invocationHandler.getClass().getName().contains("easymock");
    } catch (IllegalArgumentException e) {
        return false;
    }
}
}

Listing 3:

@Test
public void testSomething() {
    final AtomicBoolean called = new AtomicBoolean(false);
    subject.setCallback(new SomeCallback() {
        public void callback(Object arg) {
            // check arg here
            called.set(true);
        }
    });
    subject.run();
    assertTrue(called.get());
}
14
2022-07-16 15:32:07
Source

For Java, look into phase 12 of JCIP. There are some concrete instances of creating deterministic, multi - threaded device examines to at the very least examination the accuracy and also invariants of simultaneous code.

"Proving" string - safety and security with device examinations is much dicier. My idea is that this is far better offered by automated assimilation testing on a selection of platforms/configurations.

6
2022-06-30 00:51:51
Source

I such as to write 2 or even more examination approaches to execute on identical strings, and also each of them make telephone calls right into the object under examination. I've been making use of Sleep() phones call to work with the order of the telephone calls from the various strings, yet that is not actually trusted. It is additionally a whole lot slower due to the fact that you need to rest enough time that the timing generally functions.

I located the Multithreaded TC Java library from the very same team that created FindBugs. It allows you define the order of occasions without making use of Sleep(), and also it is trusted. I have not attempted it yet.

The largest constraint to this strategy is that it just allows you examine the circumstances you believe will certainly create problem. As others have actually claimed, you actually require to separate your multithreaded code right into a handful of straightforward courses to have any kind of hope of extensively examining them.

As soon as you've meticulously examined the circumstances you anticipate to create problem, an unscientific examination that tosses a number of synchronised demands at the class for some time is an excellent way to seek unanticipated problem.

Update: I've played a little bit with the Multithreaded TC Java collection, and also it functions well. I've additionally ported several of its attributes to a.NET variation I call TickingTest.

6
2022-06-08 08:31:01
Source

Pete Goodliffe has a collection on the unit testing of threaded code.

It is hard. I take the less complicated escape and also attempt to maintain the threading code extracted from the real examination. Pete does state that the means I do it is incorrect yet I've either obtained the splitting up right or I've simply been fortunate.

6
2019-12-03 05:12:21
Source

Tough one without a doubt! In my (C+npls) device examines, I've damaged this down right into numerous groups along the lines of the concurrency pattern made use of :

  1. Unit examinations for courses that run in a solitary string and also aren't string mindful - - very easy, examination customarily.

  2. Device examinations for Monitor objects (those that execute integrated approaches in the customers' string of control) that reveal an integrated public API - - instantiate numerous simulated strings that work out the API. Construct circumstances that work out inner problems of the passive object. Include one longer running examination that primarily defeats the hell from it from numerous strings for an extended period of time. This is unscientific I recognize yet it does construct self-confidence.

  3. Device examinations for Active objects (those that envelop their very own string or strings of control) - - comparable to # 2 over with variants relying on the class layout. Public API might be obstructing or non - barring, customers might get futures, information might get to lines up or require to be dequeued. There are several mixes feasible below ; white box away. Still calls for numerous simulated strings to make phone call to the object under examination.

As an apart :

In inner programmer training that I do, I educate the Pillars of Concurrency and also these 2 patterns as the key structure for thinking of and also decaying concurrency troubles. There is clearly advanced principles around yet I've located that this set of essentials aids maintain designers out of the soup. It additionally brings about code that is extra device testable, as defined over.

73
2019-12-03 04:31:37
Source

Look, there is no very easy means to do this. I'm working with a task that is naturally multithreaded. Occasions can be found in from the os and also I need to refine them simultaneously.

The most basic means to manage screening facility, multithhreaded application code is this : If its also intricate to examination, you are doing it incorrect. If you have a solitary instance that has numerous strings acting on it, and also you can not examine scenarios where these strings tip throughout each various other, after that your layout requires to be redone. Its both as straightforward and also as facility as this.

There are several means to program for multithreading that stays clear of strings going through circumstances at the very same time. The most basic is to make all your things unalterable. Certainly, that is not generally feasible. So you need to recognize those areas in your layout where strings interract with the very same instance and also lower the variety of those areas. By doing this, you separate a couple of courses where multithreading in fact takes place, lowering the total intricacy of examining your system.

Yet you need to understand that also by doing this you still can not examine every scenario where 2 strings tip on each various other. To do that, you would certainly need to run 2 strings simultaneously in the very same examination, after that regulate specifically what lines they are implementing at any kind of provided minute. The most effective you can do is imitate this scenario. Yet this could need you to code especially for screening, which goes to ideal a half action in the direction of a real remedy.

Possibly the most effective means to examine code for threading concerns is via fixed evaluation of the code. If your threaded code does not adhere to a limited set of string secure patterns, after that you could have a trouble. I think Code Analysis in VS does have some expertise of threading, yet possibly very little.

Look, as points stand presently (and also possibly will represent a great time ahead), the most effective means to examine multithreaded applications is to lower the intricacy of threaded code as high as feasible. Decrease locations where strings connect, examine as best as feasible, and also make use of code evaluation to recognize threat locations.

280
2019-12-03 04:28:04
Source

I have had the unfavorable job of screening threaded code and also they are most definitely the hardest examinations I have actually ever before created.

When creating my examinations, I made use of a mix of delegates and also occasions. Primarily it is everything about making use of PropertyNotifyChanged occasions with a WaitCallback or some sort of ConditionalWaiter that surveys.

I am not exactly sure if this was the most effective strategy, yet it has actually exercised for me.

2
2019-12-03 04:27:53
Source