First, it is important to disambiguate between concurrent multitasking and sequential multitasking (Salvucci et al, 2009). In a recent publication I present a short overview on related work of both (Jeuris and Bardram, 2016):

During concurrent multitasking cognitive resources have to be divided across several competing parallel tasks, such as driving while talking on the phone. Many ‘dual-task’ studies investigate dual-task interference and reduction of performance while performing two simultaneous tasks.  ...

In contrast to concurrent multitasking, sequential multitasking denotes the interleaving of several primary tasks which are executed one at a time. ... Studies investigating sequential multitasking are among other things interested in measuring the effects of task interleaving on productivity and accuracy. ...

Results show an inverted U-relationship between multitasking and productivity; there is thus an optimal amount of task switching which leads to the highest productivity. However, increased levels of multitasking lead to a significant loss in accuracy, indicating a trade-off between productivity and accuracy (Adler & Benbunan-Fich, 2012).

In regards to the effect of task difficulty on sequential multitasking, "subjective task difficulty has been found to be a determining factor: easy tasks benefit from multitasking by increasing stimulation, whereas hard tasks decrease performance as the result of an overload in mental workload (Adler & Benbunan-Fich, 2015)".

So to answer your question, at least one study has found that for the tasks they used more challenging tasks do 'suffer' more from multitasking. However, there is still an optimum amount of multitasking.

First, it is important to disambiguate between concurrent multitasking and sequential multitasking (Salvucci et al, 2009). In a recent publication I present a short overview on related work of both (Jeuris and Bardram, 2016):

During concurrent multitasking cognitive resources have to be divided across several competing parallel tasks, such as driving while talking on the phone. Many ‘dual-task’ studies investigate dual-task interference and reduction of performance while performing two simultaneous tasks.  ...

In contrast to concurrent multitasking, sequential multitasking denotes the interleaving of several primary tasks which are executed one at a time. ... Studies investigating sequential multitasking are among other things interested in measuring the effects of task interleaving on productivity and accuracy. ...

Results show an inverted U-relationship between multitasking and productivity; there is thus an optimal amount of task switching which leads to the highest productivity. However, increased levels of multitasking lead to a significant loss in accuracy, indicating a trade-off between productivity and accuracy (Adler & Benbunan-Fich, 2012).

In regards to the effect of task difficulty on sequential multitasking, "subjective task difficulty has been found to be a determining factor: easy tasks benefit from multitasking by increasing stimulation, whereas hard tasks decrease performance as the result of an overload in mental workload (Adler & Benbunan-Fich, 2015)".

So to answer your question, at least one study has found that for the tasks they used more challenging tasks do 'suffer' more from multitasking. However, there is still an optimum amount of multitasking.