Would you recommend converting message value using the schema before storing in S3 or PostgreSQL sinks?
It really depends on your storage pattern and partitioning scheme. Messages on Kafka topics are serialized and stored in Avro format, directly from Kafka without deserializing. If you need to partition data by some type (messagetype, symbol, etc), then you’ll need to deserialize the message. Nasdaq Data Link converts the Avro messages to Parquet in a consumer before writing to S3.