Adventures of Penrose the Mathematical Cat (1997) Theoni Pappas 
This is a book of vignettes discussing math topics rather than a novel or a true short story. The author (Pappas) uses the exploits of her cat, Penrose, to present topics in a fun way. 
The Boy Who Reversed Himself (1986) William Sleator 
It's a book catering to a preteen or early teen audience about three high school students' adventures.. 
Carry On, Mr. Bowditch (1995) Jean Lee Latham 
It is written for a younger (approximately 5th grade) audience, but it is a fine example both of mathematics and literature. 
Counting on Frank (1990) Rod Clement

Lots of people seem to really like this children's picture book about a boy who likes to ask (and answer) questions like: "How long would it take to fill up the room with water if I left the bathtub...

The Dot and the Line: A Romance in Lower Mathematics (1963) Norton Juster

This picture book describes the love story of two geometrical figures. It was also made into a cartoon by Chuck Jones. 
A Gebra Named Al (1993) Wendy Isdell

In this story, Julie falls asleep on her algebra book after spending a few frustrating minutes trying to finish her homework. An imaginary number comes to visit her in her room, and transports her to.. 
Hannah, Divided (2002) Adele Griffin

The story of a 13 year old girl living in rural Pennsylvania in 1934, "Hannah" presents us with yet another fictional account of someone who is not only talented in mathematics but also psychologically... 
The Heroic Adventures of Hercules Amsterdam (2003) Melissa Glenn Haber 
The plot focuses on a three inch tall boy who runs away from humans to live with mice, only to discover that the mice are regularly massacred by rats every seven years. 
Jayden's Rescue (2002) Vladimir Tumanov

This novel's plot revolves around mathematical puzzles for the grades 46 level. 
The Man Who Counted : A Collection of Mathematical Adventures by Malba Tahan

Delightful adventures of a medieval arabic mathematician. It is aimed at young readers (10+) but can be enjoyed by all. 
Many Moons (1943) James Thurber

In this famous children's tale about a princess who wants the moon, "the mathematician" is one of three wisemen who shows himself not to be so wise. 
Math Curse (1995) Jon Scieszka / Lane Smith (illustrator) 
In this children's picture book, the main character finds that "anything can be a math problem" when her elementary school teacher puts a math curse on her. 
The Midnighters (Series) (2004) Scott Westerfield

Teenagers discover an extra hour to the day during which they can do things while everyone else is frozen. Unfortunately, they also have to worry about the Darklings! 
The Mouse and his Child (1967) Russell Hoban 
Not really a kid's book (too violent and depressing) nor an adult book (about a toy mouse that goes on an adventure, with illustrations) this is nonetheless an interesting allegory for those so inclined... 
Nena's Math Force (2005) Susan Jarema 
This picture book for children, which is available for free online and also in print, tells the story of a girl who is upset when her math teacher requires the class to do arithmetic without a calculator. 
Number 9: The Search for the Sigma Code (1998) Cecil Balmond

A young boy learns about mathematics while trying to solve a mathematical puzzle. 
The Number Devil (Der Zahlenteufel) (1997) Hans Magnus Enzenberger 
The title may be translated as The Counting Devil, or maybe The Number Devil, and it has a subtitle that translates to 'a pillowbook for everyone who is afraid of math.' 
The Phantom Tollbooth (1961) Norton Juster 
This " Alice in Wonderland" type children's book follows our hero, Milo, to the fantasy world through his toy tollbooth. One of the lands he visits is very "mathematical." 
The Secret Life of Amanda K. Woods (1998) Ann Cameron 
A preteen novel, obscurely set in the 50s. It was a National Book Award finalist. 
The Shadow Guests (1980) Joan Aiken 
After his mother's death, a boy goes to live with his aunt, a mathematician, in her haunted English house. The mathematician is presented in a very positive light as compared to the stereotype mathematician that we have seen in so many other works of mathematical fiction. 
Sir Cumference and the ... (1997) Cindy Neuschwander 
These are pun filled picture books. 
Strange Attractors (1990) William Sleator 
Timetravel story for young adolescents with a little bit of chaotic dynamical systems thrown in. The plot follows Max, a high school student with an interest in math and science. 
Szatan Z Siodmej Klasy, Kornel Makuszynski 
Polish novel written for young adults in which one of the characters is an amateur mathematicians attempting to prove Fermat's Last Theorem. 
A Tangled Tale Lewis Carroll 
A collection of ten mathematical puzzles in story form by the famous author/mathematician Charles Dodgson (aka Lewis Carroll). "The reason I answered 3 for 'Mathematical Content' is that all the math..." 
Thomas Gray: Philosopher Cat (1988) Philip J. Davis 
You wouldn't believe it, but the famous popular math writer produced a sequel to the Oz books in which Dorothy travels to New York City through a Klein bottle (built out of two Mobius strips by the same fellow who built the Tin Man). 
A Wrinkle in Time (1962) Madeleine L'Engle 
In this classic children's adventure story, "time travel is explained as a tesseract, a five dimensional figure. By traveling along the tesseract, one bypasses the space in between." 
Hitchhiker's Guide to the Galaxy (1979) Douglas Adams 
Everyone ought to read this trilogy of four (or is it five now?) books that brilliantly combine science fiction with the drollest of British humor. 
The Light of Other Days (2000) Arthur C. Clarke / Stephen Baxter 
Using the WormCam (a camera sent through a wormhole in spacetime), it is possible to witness any event that is taking or has taken place in the universe. WormCam looks back at Pierre Fermat as he wrote the famous marginal note that inspired so much interest in the question of whether xn+yn=zn has any solutions in the positive integers for n>3.

The Story of Yung Chang (1900) Ernest Bramah (Ernest Bramah Smith) 
Before the invention of multiplication tables, a Chinese idol merchant must sell his wares individually, even if someone wishes to purchase a large amount, since he has no way to determine how much money... 
Surreal Numbers: How Two ExStudents Turned on to Pure Mathematics and Found Total Happiness Don Knuth 
The famous computer scientist (known to many grateful mathematicians as the creator of TeX) presents Conway's "surreal numbers" in the form of a fictionalized dialogue . Includes exercises! 