Your soil & region requires special management. Contact your county's Extension Service office where you'll receive the answers you need.

Locate the office with this interactive map: see link

Here is a link that might be useful: Locate your county's Extension Service office

Along with the silt flooding brought down years ago was organic matter, and that is what the soil you have needs. Since peat moss is a non renewable resource, and what I have seen of Oregon says it is not needed anyway, concentrate on available sources of organic matter, including compost. Tree leaves have the added advantage of having nutrients that peat moss lacks.

Kimmsr, thank you for the reply. I can not be sure if you are reccomending that I skip bringing the available silt to my garder and only focus on adding compst, or doing both. I have no deciduous trees on my property, and no lawn, so to add compost I have to travel a few miles to a local nursery/garden supply center and purchase it.

Horse manure on the other hand can be had for free, as can the silt. (again, the silt is on my property, only would need to shovel it into my truck and offload back into my garden plot.)

since it would indeed be a lot of hard labor to gather the silt, I do wonder if it would be usefull enough addition to my gravel/sandy volcanic pumice soil. If the addition of the silt would only be of marginal value, I would instead just purchase a few yards of compost. Or maybe the free horse manure.

One vote for silt, I add it to my compost. I do buckets from a creek bank.

What that soil needs is organic matter. Silt is just one of the mineral components of soil along with clay and sand. Importing more mineral components will do little to help while organic matter will.
Manure can help, some, but that soil also needs vegetative waste. While you may not have deciduous trees others around may have and many of those may well simply throw those leaves away. Look around to what vegetative waste might be available to use.

Both if you don't mind the labor. Hard to say what's in that silt but pretty good bet there is a good mix of all sorts of stuff in there.

Manure is rarely just manure - it is usually manure plus straw or woodchips (green plus brown = compost). If it were me (prior to lumbar spine implant, lol) I would toss in the nice wormy silt, cover it with a layer of manure compost and let the worms do their thing. If you want to plant NOW then skip the 'fresh' manure in favor of well composted stuff or it will be too hot. If you are not planting the bed this season I'd do the silt and then dump a LOT of that fresh manure compost and let it rot. Even if you are planning to plant this year then once fall gets here and the bed is 'done' for the year I would dump 3x the volume of compost it took to fill the bed and let time and worms reduce it for next year.

"Silt is just one of the mineral components of soil along with clay and sand. Importing more mineral components will do little to help while organic matter will."

I disagree. Organic matter, ultimately, turns back into CO2. Yes, in the short term it will help. Yes, if clippings or leaves are left to decay they will build up some organic matter in the soil. CO2 -> sugar -> organic mass -> decaying organic mass -> CO2. But you are in Michigan, OP is in Oregon with summer drought. Silts are more moisture retentive and will stay that way "forever" at least on a gardener's (or civilization's) timescale.

But I think applying both the silt and some composted organic matter would be a good idea. Yes, the silt probably continues humic/organic material. However, both could be somewhat tricky to integrate into the course grit. I suppose pyromanic has a tiller?

Soil particles are listed as one of these three, sand, silt, and clay. Sand is the largest of those particles with silt next in size and clay as the smallest and most uniform. They are all formed from the native rocks which determine, somewhat, what kind of soil you have.
Silt is a mineral, not vegetative material. Adding more mineral particles will not do much to change the soil and make it more productive while the vegetative waste will.

Deschutes County Extension Service has some docs specifically for Central Oregon where the soil is often thin, sandy and/or gravelly:
http://extension.oregonstate.edu/deschutes/garden-publications

Here is a link that might be useful: Gardening in Central Oregon

An article in the June 2013 issue of Fine Gardening magazine, "Clod Busters" has a side bar "strategies for 8improving clay soils" that will tell those that read it to
1. Add organic matter to improve texture.
Tilling in generous additions of compost or leaf mold will add texture and structure while aerating it and minimizing the chances of compaction.
2. Minimize traction.
Take care not to over till or step on the soil after the addition of organic matter. While tilling itself can be beneficial, the force and traction required can further compact the soil. All things in moderation.
3. Reduce tillage over time.
Once you have started to build a healthier, more porous soil environment, let nature takes its course. As you reduce tillage the ratio of fungi to bacteria increases, earthworms increase in numbers, and the organic matter further increases to stabilize soil aggregates.

There is also evidence that tilling so disturbs the soil that the Soil Food Web deteriorates. Tilling also disrupts the symbiotic relationship, mycorrhizal, that various fungi form with some plants.

Working in reverse order:
"There is also evidence that tilling so disturbs the soil that the Soil Food Web deteriorates."
http://extension.missouri.edu/p/g6955
"Surface applications of organic matter do not provide the soil aeration, moisture regulation and deep root penetration that is possible when organic matter is mixed into the soil."
This tilling naysaying seems to be a common misconception these days. Yes, farmers tilling their fields every year is a bad idea. You, home gardener, tilling a bed ONCE before planting it is not a bad idea. I have many tricky to grow rhododendrons that would otherwise be impossible on the native soil growing very well because I tilled some sphagnum moss (just a little) and a permanent structure enhancer (a lot) like Turface or Permatill into the soil. I am not the only one with this experience; I also know a dwarf conifer collection in North Carolina who would otherwise not be able to keep many of those fussy little things alive. The NCSU arboretum would also not have otherwise bought several truckloads of the stuff if it wasn't useful.

Anyhow, I agree the OP should add organic material. Definitely should. But in a dry climate situation where you have a scree like soil, organic+silty clay will retain more moisture than organic+scree alone. Obviously if they were trying to grow cacti and succulents it wouldn't make sense to do that. Apparently they are making a vegetable garden. Most vegetables like moist soils. As minitrucker said, he's done it, and I've heard and read of other people with very coarse soils adding something finer. I also know of people who have "improved" clay soils in raised beds with organic material, only to find that after about 15 years those raised bed sink back down and become even more of a quagmire than what was there before. This is in the south with higher rainfall and heat helping decay the material faster than in MI, but it can happen.

Sorry for delay in replying and thank you all for taking the time to reply. I learned a lot from all of you!

1st, yes, this is to be a vegitable gardern.

2nd, yes, I will have access to a tiller

Well, money is short, but before the snow flys, I will be bringing in probably 2, maybe 3 yards of compost from the local recycling company. Possibly before then, I may be able to aquire some horse manure.

But meanwhile, since so many were reccomending organic matter, I got to scrounging around on the property looking to see if I could find anything. I had a lot of grass clippings around and thought to start a cubic yard of hot compost, but no brown. While searching my floodplain and slough areas, I discovered many VERY old lodgepole pine stumps. These were cut over 25 years ago. They are at the edge of the derilict slough, and ALWAYS wet. I was able to harvest 3 wheelbarrows of rotten punk duff from each of these. The stuff would crumble in the hand except it damp. I have scrounged all I could find, and was able to cover my garden plot with approx 4 inches of this stuff. Hope it wasn't a mistake. Also used it as "brown" for my compost bin. (had some aspen leaves too)

And then I decided to go ahead and start bringing up some silt. After the first 6 wheelbarrow loads, I got a suprise. Under the 10 inch layer of black sandy silt, is a very deep (don't know how deep yet, I have not hit the end of it after a foot!) layer of what I think is sedge peat. It appears to be in a sandy/clay matrix. Howerver, the deep I dig, the higher the percentag of organic matter is. I did some Googling, trying to determine how nutrititious sedge peat is, and I really did not come to a conclusion. Nevertheless, the gravely, pumicy, ashy crap that I have within my fenced garden area is SO poor, I figure the combination of the black sandy silt, and the sedge peat can only be an improvement. I'm just spreading all this on top for now. Looks like I'm getting close to a 6 inch layer of the silt/sedge peat on top of the rotten pine punk/duff. I have thought to go ahead and dig it all in now, then add the compost this fall, then till it in again. Any thoughts welcome.

Peat moss does not add any nutrients to soils although they can absorb., and hold onto, those you put there.

What you're doing sounds good to me. The silt will help balance your sandy/gravelly soil to more of a loam and will help slow down the fast drainage and drying. The peat layer you found will indeed boost the organic matter content and help retain nutrients from leaching away. And finally, lots of compost (or compostables) will provide the nutrients. A multi-pronged attack!

Since you're preparing this bed for later, you can layer organic materials on it like mulch and they will decompose in place (given enough moisture and time). It might save you composting it and moving it again to spread it. You can even make layers with your silt/peat etc. right on the ground, and after a few months when you till it up and plant, well, look out, when you drop seeds you'll need to jump back so the plants don't hit you on the way up. :-]