Internal exception occurs in org.springframework.integration.dsl.filter(…) method - java

We observed DLQ messages are missing trace stack in RabbitMq when the runtime exception is originated in the execution scope of org.springframework.integration.dsl.IntegrationFlowDefinition.filter(...) method.
For example, using the following code snippet:
#Bean
IntegrationFlow accountingMovementFlow(final Function<AccountingMovement, AccountingMovement> tranMoveMapValEnricher, final MessageChannel auditDiscardChannel, final Clock clock
) {
return from("dltmvChannel")
.transform(Message.class, message -> (Message<?>) MessageBuilder.fromMessage(message)
.setHeader(SINK_STARTED_TIMESTAMP, LocalDateTime.ofInstant(clock.instant(), ZoneId.of("UTC")).toString())
.build())
.transform(new AccountingMovementMapper())
.filter(AccountingMovement::isValidOperationType,
discardFlow -> discardFlow.discardChannel(auditDiscardChannel))
.filter(AccountingMovement.class, accountingMovement -> accountingMovement.getAccountBusinessDate().isBefore(accountingMovement.getMovementEffectiveDate()),
discardFlow -> discardFlow.discardChannel(auditDiscardChannel))
.transform(tranMoveMapValEnricher::apply)
.filter(AccountingMovement.class, accountingMovement -> !accountingMovement.isExistingMovement(),
discardFlow -> discardFlow.discardChannel(auditDiscardChannel))
.channel("accountingMovementEnricherChannel")
.get();
}
Let's say AccountBusinessDate is null. Executing accountingMovement.getAccountBusinessDate().isBefore(accountingMovement.getMovementEffectiveDate()) will immediately trigger a NullPointerException. In this case, the message which flows into "dltmvChannel" will get stored in the DLQ with trace stack. However, this is not happening.
We traced/debugged the .filter(...) method and found the internal execution actually triggers another parse exception as the original runtime exception is bubbling up. As a result, the DLQ'ed message is missing the trace stack.
We don't see this behavior in the other methods of org.springframework.integration.dsl.IntegrationFlowDefinition. For example, if tranMoveMapValEnricher::apply triggers a runtime exception in while executing org.springframework.integration.dsl.IntegrationFlowDefinition.transform(...), the message gets DLQ'ed with trace stack.

Related

Spring Integration null originalMessage in ErrorMessage

We are really confused with error handling in Spring Integration. We are using Boot 2.0.2 and Kotlin.
Within a #Transformer we throw an Exception X.
Also, within a Java DSL flow definition, we throw the same exception X
Channels
#Bean(TO_BE_PROCESSED_CHANNEL)
fun toBeProcessed() = PublishSubscribeChannel(defaultExecutor())
Flow
#Configuration
class VideoDescriptorPersistSubflow(
val videoRepository: JdbcVideoRepository,
val ingestRecordRepository: CustomIngestRecordRepository
) {
#Bean
fun videoDescriptorPersistFlow(
toBeProcessed: MessageChannel,
processedVideos: MessageChannel
) =
IntegrationFlows.from(toBeProcessed)
.filter { message: Message<*> -> message.ingestRecordId() != null }
.handle { videoDescriptor: VideoDescriptor, _ -> validateVideoDescriptor(videoDescriptor) }
.handle { videoDescriptor: VideoDescriptor, _ -> videoRepository.persist(videoDescriptor) }
.channel(processedVideos)
.get()
fun validateVideoDescriptor(videoDescriptor: VideoDescriptor): VideoDescriptor {
val errors = VideoDescriptorValidator().validate(videoDescriptor)
if (errors.isNotEmpty()) {
throw VideoMetadataValidationException(errors)
}
return videoDescriptor
}
Later in the errorChannel we filter out X and do some stuff. At that point we need the failed message.
For the exception thrown by the #Transformer the originalMessage is there.
For the one thrown from the java DSL subflow, the originalMessage is null.
We did some digging and realised, the former is wrapped in MessagingExceptionWrapper whereas the later is wrapped in a MessageHandlingException which does not contain a reference to the original message.
Could someone please help us understand under what circumstances Spring Integration wraps using what exceptions? Docs do not say much here, or we couldn't find anything relevant.
UPDATE: CHANGING FROM PUBSUB TO QUEUE CHANNEL MAKES IT WORK...
UPDATE 2: Following Gary's advide we are now using payload.failed message, which works just fine. There is something dodgy though with originalMessage in ErrorMessage.
payload.failedMessage is the message at the time of failure. ErrorMessage.originalMessage is the message at the start of the flow. It is not populated in all cases.

throw multiple errors in reactor 3

Flux.create(sink -> {
sink.error(new IOException());
sink.error(new IOException());
}).subscribe(System.out::println, System.out::println, System.out::println);
I thought (wrongly) that after the first error, the flux will be disposing and not listen to any more signals but the above code creates an exception:
java.io.IOException
Exception in thread "main" reactor.core.Exceptions$BubblingException: java.io.IOException
at reactor.core.Exceptions.bubble(Exceptions.java:154)
at reactor.core.publisher.Operators.onErrorDropped(Operators.java:256)
at reactor.core.publisher.FluxCreate$SerializedSink.error(FluxCreate.java:172)
at main.App.lambda$f1$0(App.java:30)
at reactor.core.publisher.FluxCreate.subscribe(FluxCreate.java:92)
at reactor.core.publisher.Flux.subscribe(Flux.java:6447)
at reactor.core.publisher.Flux.subscribeWith(Flux.java:6614)
at reactor.core.publisher.Flux.subscribe(Flux.java:6440)
at reactor.core.publisher.Flux.subscribe(Flux.java:6404)
at main.App.f1(App.java:31)
at main.App.main(App.java:23)
Caused by: java.io.IOException
... 8 more
Why I got this exception?
As per reactive streams it is illegal to signal onError() twice, and when the create operator receives the second exception, it drops it. There is a hook that can be used to capture such cases and do something meaningful with it, but in your case the default implementation kicks in, which is simply to wrap the extraneous exception and throw it, or "bubble" it up, to the caller.

Reactive patterns with argument preconditions

I'm working with RxJava and I don't know where the right place to check arguments would be. For example, say I have the following:
public Completable createDirectory(final String path) {
return Completable.create(emitter -> {
final File directory = new File(path);
final boolean createdSuccessfully = directory.mkDirs();
if (createdSuccessfully) {
emitter.onComplete();
} else {
emitter.onError(new IOException("Failed to create directory.");
}
}
}
Would it be better to check for a null path in the root of the method, or at the start of the completable? I'm leaning towards the former, but I'm interested in the pros and cons of both approaches.
I would say: it depends on what you are trying to achieve.
If you check for null-value on method-entry, before Completeable.create, you would throw the NPE/ IllegalArgumentException on calling-thread. This would follow the 'fail fast'-philosophy. Possible pros:
* fails fast on method-invocation
* Exception in callstack of calling-thread
If you check in Completable.create-lambda, it will be invoked on lambda-evaluation (subscription).
Example: Call createDirectory-Method on calling-thread, add subscribeOn and subscribe: NPE / IllegalArgumentException will be thrown on subscribeOn-Scheduler-Thread on subscription.
It will only be thrown, when you subscribe to returned Completable. Maybe you create an Completable but never subscribe to it. Then no exception will be thrown. This could be a pro for checking in lambda. If you would not check, a NPE would be thrown anyways on subscribeOn thread. Exception thrown on subscribeOn thread could be negative, because you only see the trace for subscribedOn-thread. Sometimes it is not easy to see the flow, when switching threads with only on callstack. You would not see that createDirectory-method was invoked. You would only see the lambda invocation plus some reactive-stack-polution.
I would say: use fail fast, if you subscribe to created Completable at some point. If it is possible, that no subscription to created Completable happens at any time, it would probably use some Precondition with a message, because it will only throw, when subscribed to (lazy). Anyways, if you check with Objects#requireNonNull() without any message in the lambda, you could just discard checking for null, because the NPE will be thrown anyway.

Akka Supervisor Strategy for Router

I am very new to Akka and I had the following question:
I have a MasterActor where a router is defined and that is where I defined the supervisionStrategy:
override val supervisorStrategy = OneForOneStrategy() {
case _: FileNotFoundException =>
println("****** Failed actor in FileNotFound: " + sender)
Restart
}
I then passed this supervisionStrategy to the router:
val router = context.actorOf(RemoteRouterConfig(RoundRobinPool(3, supervisorStrategy=supervisorStrategy), addresses).props(Props(classOf[MapActor], reduceActor)), "router")
I placed a print statement in the postRestart method and I see that only one actor is restarted. This led me to believe that I was on the right track.
However, I have 2 issues so far:
Issue #1:
I noticed that sender is deadLetters in the print statement. I expected it to be the failed actor.
Issue #2:
I put a print statement in the postRestart method and I saw that only one routee is restarted when a FileNotFound exception is encountered and the processing of the continues. That seems like the correct behavior.
However, if I change the supervisorStrategy to Stop on the FileNotFound exception, (shown below), I see that ALL actors stop and I do not ANY processing taking place, at all. I expected to see processing occur by all actors until one encounters a FileNotFound exception. When the failed actor hits the FileNotFound exception, I thought that ONLY that actor should be stopped and the others should continue processing.
override val supervisorStrategy = OneForOneStrategy() {
case _: FileNotFoundException =>
println("****** Failed actor in FileNotFound: " + sender)
Stop
}
With respect to issue #1: this is expected. The sender() you're calling here is the one in the context of your MasterActor. It is only allowed to call this method during message processing, so behavior in this example is undefined. (The reference to deadLetters here is confusing, this has been improved in version 2.5.1 of Akka)
With respect to issue #2 I'm not quite sure, I suspect it has to do with the fact that you're wrapping the RoundRobinPool in a RemoteRouterConfig but I'm not that familiar with that part of the system.

Observable's doOnError correct location

I am kind of new to Observers, and I am still trying to figure them out. I have the following piece of code:
observableKafka.getRealTimeEvents()
.filter(this::isTrackedAccount)
.filter(e -> LedgerMapper.isDepositOrClosedTrade((Transaction) e.getPayload()))
.map(ledgerMapper::mapLedgerTransaction)
.map(offerCache::addTransaction)
.filter(offer -> offer != null) // Offer may have been removed from cache since last check
.filter(Offer::isReady)
.doOnError(throwable -> {
LOG.info("Exception thrown on realtime events");
})
.forEach(awardChecker::awardFailOrIgnore);
getRealTimeEvents() returns an Observable<Event>.
Does the location of .doOnError matters? Also, what is the effect of adding more than one call to it in this piece of code? I have realised I can do it and all of them get invoked, but I am not sure of what could be its purpose.
Yes, it does. doOnError acts when an error is passing through the stream at that specific point, so if the operator(s) before doOnError throw(s), your action will be called. However, if you place the doOnError further up, it may or may not be called depending on what downstream operators are in the chain.
Given
Observer<Object> ignore = new Observer<Object>() {
#Override public void onCompleted() {
}
#Override public void onError(Throwable e) {
}
#Override public void onNext(Object t) {
}
};
For example, the following code will always call doOnError:
Observable.<Object>error(new Exception()).doOnError(e -> log(e)).subscribe(ignore);
However, this code won't:
Observable.just(1).doOnError(e -> log(e))
.flatMap(v -> Observable.<Integer>error(new Exception())).subscribe(ignore);
Most operators will bounce back exceptions that originate downstream.
Adding multipe doOnError is viable if you transform an exception via onErrorResumeNext or onExceptionResumeNext:
Observable.<Object>error(new RuntimeException())
.doOnError(e -> log(e))
.onErrorResumeNext(Observable.<Object>error(new IllegalStateException()))
.doOnError(e -> log(e)).subscribe(ignore);
otherwise, you'd log the same exception at multiple locations of the chain.
the doOn??? methods are there for side-effects, processing that doesn't really is your core business value let's say. Logging is a perfectly fine use for that.
That said, sometimes you want to do something more meaningful with an error, like retrying, or displaying a message to a user, etc... For these cases, the "rx" way would be to process the error in a subscribe call.
doOnError (and the other doOn methods) wraps the original Observable into a new one and adds behavior to it (around its onError method, obviously). That's why you can call it many times. Also one benefit of being able to call it anywhere in the chain is that you can access errors that would otherwise be hidden from the consumer of the stream (the Subscriber), for instance because there's a retry down in the chain...

Resources