You can just buy the microcontroller and do it yourself from there. Here is a nice example someone did of a simple LED flasher  with just 6 parts:
1. An ATmega ATTiny85 microcontroller
2. A socket for that processor
3. A coin cell battery
4. A holder for the battery
5. A resistor
6. An LED
and some wire and solder.
What going with an actual Arduino or Arduino compatible gets you, from a hardware point of view, is a bunch of ready made attachments. For instance, suppose you have some sensor that needs an odd voltage and has weird timing requirements. It will be a lot more convenient to get a shield that has that sensor, and a voltage converter, and something that deals with the weird timing and presents a simple I2C interface to your code than to have to do all that yourself.
There are some EdX courses that you might find useful.
From UTAustinX, "Embedded Systems--Shape the World" . This is a lab-based course where you do 13 or so labs using a TI Tiva Series C Launchpad. That's an 80 MHz ARM Cortex M4 board. Cost for the hardware for the course is $35-$55, depending on if you want to do a couple of the optional labs.
From UCBerkeleyX, "Electronic Interfaces: Bridging the Physical and Digital Worlds" . Another lab course
From MITx, "Circuits and Electronics" . The online version of MITs 6.002 introductory electronics course.
I am looking forward to two embedded system courses in EdX.
Electronic Interfaces: Bridging the Physical and Digital Worlds: https://www.edx.org/course/electronic-interfaces-bridging-ph...
Embedded Systems - Shape The World: https://www.edx.org/course/embedded-systems-shape-world-utau...
EdX has got some nice electronics courses: https://www.edx.org/course
Between, I am from compsci background