When trying a drug case with a search warrant, is the search warrant admissible into evidence before the jury?

If the Defendant is challenging the admissibility of the fruits of the search, I would say yes, but never just to prove the truth of the matters asserted in the affidavit.

The warrant itself shouldn't be admissible. How can the defendant challenge it or the contents in front of the jury?

The legality of the warrant is a decision for the judge. There shouldn't be any factual dispute about its contents. It says what it says, so there is no fact dispute for the jury to decide.

An officer certainly could testify that he obtained a warrant, and the warrant itself could be admitted to prove that fact, but not the contents of the affidavit, which are hearsay.

I don't think the affidavit is admissible. It's usually filled with hearsay and sometimes uncorroborated information. On the other hand, I usually offer the warrant without the affidavit just to explain to the jury the legal authority to conduct the search. The officer can also testify that he obtained a warrant, how he did it, etc. I suppose the defense could object but I don't see why anyone would have a problem with just introducing the warrant.