Request - encrypted cache « OneSwarm Forum

OneSwarm Forum » Feature Requests

Request - encrypted cache

(20 posts) (10 voices)

Started 4 years ago by ksork
Latest reply from Natanael_L

Tags:

No tags yet.

ksork
Member

Hi

I have tried another p2p SW for anonymous file sharing, perfect dark (PD). The PD has one rather nice feature, the encrypted cache. I am not sure if that concept is in line with the oneswarm's concept, but if so, it's a rather nice feature.

The concept is that you must have x GB encrypted cache that PD will fill automatically to x GB before you can start to download. All upload is made into the cache. All DL is made from the cache. You can therefore never-ever se what data user X that is connected to you is DL from you. It can be;
+A file in your cache that you are sharing
+Data that just is routed trough you
+Data in the cache that is file "y", but you are not aware that you have it.

One "must have" feature in the cache is a "block/ban" list for unwanted filenames (or part of filename), to avoid material as CP to be shared.

One key benefit from the cache is also the longlivity of material, no-one can remove material that is old etc (except for files with names on blocklist).

I don’t know if such a cache is in line with the usage of oneswarm, but it works rather well in PD. This cache would ofcurse work in parallel to the direct share to your friends (but some of the files in the direct share may end up in the encrypted cache) depending on implementation.

Well, it was just an idea that I wanted to share ;-)

BR / K.sork

Posted 4 years ago #
DarkAlf
Member

I know this is an old post. But I really think this is a good idea. Two reasons:

1) Longevity of files.
2) Solving the issue with downloading above your upload. http://forum.oneswarm.org/topic/532

Posted 4 years ago #
Natanael_L
Member

This would just make OneSwarm freenet-ish

---------

How would the downloader decrypt it?

Posted 4 years ago #
isdal
Administrator

One of the reasons we are a bit reluctant to add a cache like the one you describe is that many users might feel uncomfortable with having stuff on their HD that they don't really have control over. Our current view is that in OneSwarm people only store files that they either downloaded or shared which makes it easier for users to reason about what is going on, other systems (like freenet) take a different approach and there are advantages and disadvantages with both.

Posted 4 years ago #
MaTee
Member

I'm not uncomfortable with a cache. It could be optional, and turned off as a default.

When a non UI version comes out, i will run a server on a fast (1 Gbit/s) connection, and it would be nice if i can use some disk space to speed things up and make it more robust.

The data going through the server is often popular, and it's already using my bandwidth anyway.

But i guess it's a lot of work to implement this..

Posted 4 years ago #
ksork
Member

-> Isdal
You have a point in that.

Posted 4 years ago #
ps
Member

a cache could be a nice thing for files which are heavily requested, .. but why does it have to be encrypted? Either way, in a cache you would typically not find complete files (if they are larger) but chunks only.

And hey.. oneswarm wants to be a f2f program where files are transferred between friends. Another reason why there is no need of encrypting the cache.

Posted 4 years ago #
Natanael_L
Member

@ps: But you are relaying stuff that goes trough your friends without knowing if it's from them or from others.

How can you know it ain't kid porno from your friends friends friends friends huge-mass-imported-friend-list friend from that key-sharing site? Wanna cahce that? Encrypted, so that you can't know you are relaying it?

Posted 4 years ago #
ksork
Member

Hi

The idea with the cache is that you also must have a possiblity to "blacklist" some words (eg. kid porno) that may be used to find that kind of porno.

The main upside with the cache is the longlivity of files.

Posted 4 years ago #
ksork
Member

But the main purpose with the cache is that a seeder never ever knows what a DL is reciving. An exampel;
You are a goverment organisation that wants to "catch" someone. You share file X. You must, with the cache, add X to the cache. X is 1Gb large, but you must have a cache that are 4GB big before you can seed anything. The SW must therefor reecive additional 3GB from friends (it does that automaticly). What is the other 3GB ? You dont know. You seed somthing to friend_1. But what is it you share ? file X or somthing from the other 3GB, well, you dont know....

Posted 4 years ago #
ps
Member

@Natanael_L

another point in favor of a non-encrypted cache. I could see whats being relayed and I could delete it if I don't agree with the content.

Posted 4 years ago #
Natanael_L
Member

@ps: I would recommend an automatic cache *of your friends public files*.
THAT would be useful.
And if you would want to download a cached file, OneSwarm would say "It's already cached on your machine, where do you want it copied to?"

Posted 3 years ago #
tudza
Member

So some people using Share were nailed by Japanese police:

http://www.animenewsnetwork.com/news/2009-11-30/10-arrested-in-japan-for-uploading-via-share-program

As I understand it, Share was supposed to fix a problem in the forums program used in Winny. Sounds like major fail.

Somebody else is working on yet another program called Perfect Dark.

Why are these people messing around when Freenet exists, and they know about it, or when there is such a thing as OneSwarm?

Posted 3 years ago #
lah
Member

ksork: The client software know what it's sharing from the cache - that means the client know it! Making network security depend on client software integrity is a mayor FAIL.

Some opt in caching could be useful thou. A 'cache and proxy' setting per user that make a node list that users open listed contents as it's own to others and cache it if someone d/l it (user or from the network). Would be great for 'headless cooperative 1s nodes' where all users (member friends) can see all other users content without knowing who it is from (except if they control the cooperative node). All that content will also be shared on the network improving longlivity.

Thinking 50 users sharing a 4TB 24/7 nod with 1 Gbit network and thousands of limited friends. They should also be able to use the node directly by limited remote access (that don't reveal from who different files is shared) for mobile viewing. Stuff downloaded directly will also be cached. No delete function in limited mode - all deletes is least recently used data. Maybe a feature to flag bad data and if x users flag it it will be deleted (and blocked from proxying).

Some hundreds of nodes like that will certainly help the network :-)

Being strictly a cashing service it is at least in an legal grayzone - not clearly illegal.

Posted 3 years ago #
cronborg
Member

@ isdal.
Cache wouldn't be great for Oneswarm. This must be explained.

Systems based on cache may have huge latency and redundant data emitted. This is kind of different network. It's much more secure if it's well done, but there is at best so much latency you need to wait much more to get that data, because you won't get it at once since that would reveal all the network even better than with Oneswarm real time transfers. Furthermore that would imply that all the protocol should move to that solution, because then, the way to make that thing work is to upload the data you want to share on the "global cache" that would have to be redundant, and after that people interested in that data would have to download from the cache. This is 2 or 3 times more traffic for the same purpose, if everybody plays by the rules, waits for the upload to finish, creates huge amount of cache to help the network. I don't believe this brings so much better security and anonymity to enjoy the so much smaller efficiency.

Posted 3 years ago #
cronborg
Member

Share, perfect dark... all those are Freenet/Darknet made simple and less secured.
The darknet use of Freenet is kind of state of art of anonymity and security but you don't get the expectations of a standard p2p system efficiency by far. It's slooooow and most of the times it never happens.

Posted 3 years ago #
Natanael_L
Member

The best way to implement cache would be to just let 1S use unused bandwidth to fetch your friends' public files and store them, as I mentioned before, as well as to keep track of file requests passing through the 1S client and store files that pass through it the 3rd time it sees the same file.

That means that is your friend has Game.exe publicly visible, your 1S client will fetch and store it when it has unused bandwidth and HDD space for it.
And if friend 3 sends a request for file X on friend 4's 1S client PC, your 1S node will cache it the 3rd time. So that when friend 5 sends the 4th request for that file, it will be fetched from your cache instead of your 1S client relaying it from friend 4.

Cache method #1 here will increase bandwidth use initially, but only the unused bandwidth.
Cache method #2 will decrease bandwith usage since popular files will pass through fewer nodes after the 3rd request will and be stored in more places and thus closer to each node in average. There will not be any extra file requests since 1S only acting like a HTTP proxy with cache, but for 1S instead.

Posted 3 years ago #
ahiramazu
Member

I don't think content caching would be a good idea for default installs. Perhaps as an optional plugin that dedicated users with enormous bandwidth can install. Not a priority.

On the other hand I think search result caching could be a very good idea if it is possible. This should make remote content more accessible and less hit-and-miss and perhaps make search results appear much quicker. Results should be ranked using some last-seen and availability rating and low grade hits (old/unreliable) should be purged when the cache is full. The question is how to implement this without opening new attack vectors.

Posted 3 years ago #
lah
Member

Natanael_L: I think 1s can do lot better then just looking on frequency.

First crossroads: Whenever a node send/receive the same swarm from more than two neighbors start a 'torrent proxy' that cache what it transfer and anons both what it self have and what other neighbors have.

Then a hash cache: compare the swarms hash with he nodes hash. If they are close cash the data transfered. low bandwidth nodes can cash allot, high bandwidth nodes less (will else eat disc i/o).

Expiring: Both how recent accessed and how close the hash match is should be considered. So swarms close to the nodes hash is cached longer. This will create a distributed cash that can hold allot of content and distribute traffic so those cashing get ratio love for it. Excessive big swarms (>10GB?) should be expired faster to not push to much off the cache.

Asymmetric flooding: If theres neighbors who's hash is close to the hash looked for flood them with less delay. This will make it more likely paths go through nodes that will cache it and cache it for long time. It also makes it more likely to find cached data. To things working great together!

I think caching should be opt in, and the size of the cache a free setting (default cache size of 0 solve both things). Caching will increase ratio and bigger cache will give better ratio so there is incitement enough to enable caching.

Think it can do small wonders. Take an average of 50 friends and an average of half using caching:

2 hopes: 2500 nodes, 1250 caching, 12 within 1/100 hashmatch, 1 within 1/1000 hashmatch.
3 hopes: 125 000 nodes, 62 500 caching, 62 within 1/1000 hashmatch, 6 within 1/10 000 hashmatch
4 hopes: 6m nodes, 3m caching, 30 within 1/100 000 hashmatch, 3 within 1/million hashmatch

It will probably not get meaningfully populated deeper than this without speculative copying and that is as cronborg warn often more cost than benefit.

Pretty good anyway. If people in average set 50GB for caching and 10GB is used for the closest hashmatches we have a cooperative cache on ~10 000 TB within 4 hopes.

Asymmetric flooding will also make us reach that cache with short latency and low flooding impact on the overall network.

Posted 3 years ago #
Natanael_L
Member

That idea looks a little better. :)

Posted 3 years ago #

RSS feed for this topic

Reply

You must log in to post.