Hello cyberpals.
I've been playing around lately with some development
of... something... on my Mac that
is not diff or Carl.
And I ran into a situation where I need
SSL or TLS and I looked
into like porting, the smallest
SSL implementation that I could find, which appears to be like
Mbed TLS or BearSSL.
And it's just not going to work.
BearSSL uses 64-bit integers which are not supported.
And I
think someone did the numbers and like on the 68k
processor to do a TLS handshake would
take like 70 seconds or something ridiculous.
And by then your-- you know, the user is
not going to wait 70 seconds and the server is not going to wait that
long either. It just time out, and the whole thing would be pointless.
So I bought a Teensy 4.0
device, which is a tiny little ARM board
that has like a I think it's 400 megahertz or
something processor on a little tiny board.
And my intention was that this thing could run BearSSL
because there's already an existing port of
BearSSL to the
Arduino platform, I think.
And so my intention was I could just hang
this little device off of the serial
port, or I guess it'd be the printer report because the serial port is in
use by the modem. I mean, they're both serial ports,
so it's either modem or printer port, and then my software
could talk to that device and then
basically relay communications between
the modem and this device. And so I would send it would
receive encrypted data over that modem and then
hand it off to this device, which would decrypt it and
then send back plaintext data to the...
to the Mac. And the more I
thought about it and started working on it, the more I realized that that's kind
of a dumb solution because every software
piece of software that wants to do TLS stuff would have to
integrate with this device.
And it's going to be very slow because it's
going to have to be shuffling all this serial data around.
And the Mac serial ports are not that fast.
So then I started thinking I could do something on
my network, which I already basically have, the way that
Carl is talking to my IMAP server is it
makes a plaintext TCP connection to
a machine on my network
that is running socat. And then socat basically access
a proxy and then make an SSL connection
to my mail server and then decrypts it and then talks
plaintext to my Mac.
So that's worked fine so far. But the problem
is that I want to be able to make arbitrary TCP
connections to arbitrary services that are running SSL.
And I can't do that with socat because you
have to basically set up socat on the server to be running,
and it has to know which server to connect to.
So then I started thinking, well, you could do SOCKS5,
and SOCKS5 is a protocol,
a standard protocol for proxy server.
So you basically make a TCP connection to your SOCKS server
on your network. You send it some custom binary data
that says, I want to make a connection to this IP or this host
name on this port, and then the server either connects
directly to that IP or makes-- does the
DNS lookup of the name and then connects to it.
But the way that most SOCKS proxies work is that
if you make a sock connection to a TLS server,
the SOCKS proxy just proxies
the encrypted data to you, and then the client is expected
to do its normal TLS handshake and verify the certificate
and everything, so that the
connection is still encrypted all the way to the end.
So what I did was I wrote my
own SOCKS proxy in Ruby yesterday that does
that, except if you ask it to connect to any IP on
a port that is in a list inside
the software, which is basically 443 which is https,
993, which is IMAPS
and 995 is POP3S. If you ask
it to make a connection on one of those ports,
the SOCKS proxy does the TLS connection
and negotiation and decryption and hands off plaintext
data back to the client. So the client
doesn't need to know anything about TLS. It thinks
that it's still talking plaintext to a remote server.
You would just have to change the port that it's talking.
So instead of talking on port 80, you talk on port 443.
So that is all working.
And the SOCKS proxy does
the SSL verification, and if the certificate fails,
it just doesn't open a connection just as if the remote
IP was not responding. So with that up
and running, what I would like to do is implement
SOCKS5 proxy support in Carl.
And so instead of having
Carl talking directly to this
server on my network with like,
a hard coded name or an IP, I could
just put in the name of my actual IMAP server,
but then had a setting
for the SOCKS proxy IP. And then if
that setting is configured, then Carl would talk
to the SOCKS proxy instead of directly to the IP. So that,
I guess, is what we're going to do today. It shouldn't take
too long because SOCKS5 is very small protocol.
So I think the first thing we need to do is add
a setting for the SOCKS proxy IP and port.
It defaults to port 1080.
Then we will modify the IMAP
or TCP code to handle that.
So that would be in STR I guess.
And then we'll
make a new one for... what are we going to call it?
We'll call it-- well, this is just
the data. So I'm going to configure 3.1
is my server.
And we need to
remember that and give it a name that
would be 1005.
I don't know if most software-- so my
reasoning for doing this is that I think even outside
of this Mac project, if you're using like an old computer
with like an old version of, I don't know, Windows or some
web browser that software might already support SOCKS, but it
doesn't support TLS or if it does, it doesn't support like TLS
1.2 or anything with like a modern cipher.
So this code might be helpful for you
as well. If you're in that situation, it's on GitHub.
It's called sockhole, the name that I gave this silly
project. So github.com/jcs/sockhole.
And then if this is set and not
blank we will use it and
then the default port is 1080 and
We will call it 1006.
and SOCKS_PROXY_PORT.
sockhole only supports SOCKS5 because SOCKS5 is needed
to give it a name and have the server
do the DNS resolution. I think SOCKS4
only supports giving it an IP and
you need the name. The server needs to know the name to do proper
TLS validation.
So I was looking at like, I have Eudora POP3 client on here,
but it doesn't support SOCKS-- any version of SOCKS
proxies. So I guess if somebody
needs an older version of SOCKS support and
then you I guess just have to forego the certificate
validation, you can implement that
in the proxy. Alright, so we have SOCKS_PROXY_IP
and PORT is 1005 and 1006.
And then again these all need to be moved
to the preferences window inside of Carl, but for
now we will say these are global
settings so they don't really belong in the IMAP
code and
I've already forgotten them because I have
a bad short term memory.
So I was looking at this thing called Gemini.
It's basically a new
protocol that's like a stripped down-- 1005
in 1,006--
like there's Gopher, Gopher is a really old text-only protocol then
there's obviously HTTP, so Gemini is
trying to position itself in between those by
making like, so you have
like a web browser that can have links and
stuff, but the amount of text
formatting that it supports is very limited so that you can't--
the idea is like if you impose all these
restrictions on what you can do with it, then it becomes harder
to like mess it up with ads
and all kinds of other crap. But the weird
thing about it-- so it's like a really low bandwidth
protocol, but then they made a requirement that everything has support TLS.
So I was looking at what it would take to make a browser on
System 6 that could support Gemini, but you'd have to do
do the TLS stuff. So I don't know
why they made that restriction, but anyway.
Alright, so now we need to look these up and then store
them, I guess. Probably in util I think I'm
storing things and
then we'll just save that as like a handle,
I guess.
Or maybe just a C string because
it won't be that big in memory.
Alright. So we will look these up
at startup.
I guess that would be an unsigned short.
Alright. And then we will look these up
here and then--
I should probably--
deal with--
well, I guess the setting is always there. It just might
be blank. So that's okay.
So we'll call this... what?
It should also be ip because
we're not going to be doing DNS resolution. I don't know if that was a
thing, like if people
would configure the network to talk to a proxy by its name.
We're going to limit this to an IP for now.
And then we'll say...
If it's zero, there's no point in keeping it.
I don't know why I did that instead of using this.
But we're going to use that.
Okay, so I
guess when IMAP does a connection,
it resolves the
name. Cause if it's
using a SOCKS proxy, it's probably better if we don't
resolve the name incase the proxy has
something weird configured and wants to remap that name.
So get all
of this.
So we
TCPCreate and then...
Alright, so maybe we can reuse that
and just set
the IP to the SOCKS proxy IP and
then maybe change that to print...
Yeah. Alright. So we will short
circuit all of that.
If we have a socks_proxy_ip,
and then we will just hand that directly to the
This can probably
be moved up
This too, I guess.
TCPCreate just like allocates like
maps the connection to these buffers without
actually connecting to anything yet.
Alright, so we do,
Wow.
Okay. So browser_status
And then we need to...
this should probably go...
in tcp somewhere.
What do you think?
Like instead of TCPActiveOpen do like
a SOCKSTCPActiveOpen.
Yeah let's do that in case we need to reuse it or something.
So this tcp is--
these are like wrapper functions around MacTCP
but it's good enough.
It's why they all have leading underscore names.
Okay, so we'll make our
own function and we'll call it-- what does
ActiveOpen return?
So we will call this SOCKS5TCPActiveOpen
And we'll just assume the SOCKS server will
resolve if we give it
an IP because I think most of
them do that.
And then I
guess we can just pass this anyway.
Okie dokie. So that is our
C prototype.
Right? So this is expecting an IP address...
but we only have a name.
I guess we shouldn't call this imap_ip.
So socksIP, socksPort,
remoteHost.
[ reading ]
Okay. So that
should work.
Okily dokily.
And then we just need to actually make this do something.
I guess this is violating my naming convention as
it should be lower case,
but we're kind of emulating a system call.
[ narrating ]
So we're connected to the SOCKS server
and we just need to implement
the actual protocol, which is...
RFC 1928.
So you basically just send it like a
protocol-- like a version, and authentication header.
The server tries to negotiate that,
and then once that's okay, you send
it what you want to
do, which is connect and then you give it the hostname and port.
The data[0] is going to be the
version which is 5.
I guess we should define this or something to be good boys.
So that would be like...
And then we need a method auth.
We need a connect.
We can also do like a bind or UDP
transmission, but we're just going to do connect,
which is 1 and
we'll send it a hostname
And if we don't get a success, we don't really care what the other
error code is. We're just going to fail. Okay.
So first
one will be...
and then there is a...
so you send it a version, the number of method-- number of auth methods,
and then the method itself.
So we're just going to say 1
is how many and
then the first method is AUTH_NONE.
And then we send that.
to the server.
It should reply that
it's okay. And then we can send the actual hostname and port.
Okay.
I probably
need to call this in a loop and then wait for the data to be available.
So basically we just want to spin until we
have some data.
How much data do we need back?
Two bytes, I think.
Let's just check.
Yeah, the server responds with the version and the
method that it's selected, I guess, which is zero.
I don't know why we'd have more than two, but we
need to read this.
Let's do this.
So that will just loop forever until I either
get a status that the connection died or something.
So if we're out of the loop, we have data available and it's more than--
greater than or equal to two bytes.
So we're going to read
from there on the stream and stick
it in there
and we want two bytes.
I'm not sure if it has to be cast to a pointer type data
We'll use our old friend, goto.
Alright, so that receives two bytes if
data[0] is not equal to SOCKS_VERSION 5.
Is it the auth type
on the number?
"The server selects from one of the methods given in methods and sends
a method selection message.
If the selected method is FF, none of the
methods listed by the client are acceptable and the client must
close the connection.
The values currently defined for method are--"
Oh right. So it
should be the type.
goto fail was the name of that Apple
security library thing bug where they weren't
validating TLS certs properly and
because there were two goto fails.
Alright, so we received the data. We verify
that the version and auth are correct and now we can just send the--
Let's make this bigger because it has to encompass remote host.
How big can remote host be?
We'll say 255.
Because why not?
And if you pass in...
we'll do this later because we need to compute the length of it anyway and
there's extra data in there. So we'll figure it out later.
data[0] will be VERSION.
data[1] will be CONNECT.
data[2] is a reserved byte.
We'll use the actual thing in the spec, a domain name.
You can also pass it an IPv4 or IPv6 address.
"If the address field contains a fully qualified domain name,
the first octet of the address field contains the number of octets
of the name that follows. There is no
terminating null octet."
[ narrating ]
[ still narrating ]
I do not like those kinds of variables.
[ reading ]
And then there's the port
which can be in 2--
2 bytes. We'll look that up because
we get it as a short.
It's probably short.
This gross include file.
I guess it's somewhere else.
I don't know what else it possibly could have been.
So, what is that like, port?
Yeah, I don't think--
it's already messing me up.
[ reading ]
And what is that? I think if you shift it...
Sorry.
Shift it rightward 8 and
then mask it... I think
that's right.
All right. And then we would send it...
Oh, we never sent this.
[ reading ]
I remember this. This took me a long time to figure out because
none of this crap is documented the right way.
The length will be two,
and we will send it...
Then we need to do the same thing.
Although we probably don't need to--
I don't know why I'm
not passing that as a pointer.
It seems like something I should do because it is not a pointer
already.
Alright, so then here we
just need to send it again and
the length is...
Right? That's the cool thing to do.
Okay. And then once
it's opened, we should get a status response,
which is just a reply. I mean
a reply-- it's just a successful status.
Which looks like...
So if we have more than
[ counting ]
I guess that should
be for the response.
So we will say--
[ mumbling ]
We don't really care what the actual code is.
And then if we're done, we can just return
zero and then the socket is open and a state
ready for sending.
So let us try that.
Like a bunch of different reply codes, like connection
not allowed by rule set, network unreachable,
host and reachable connection refused. We don't really care
as long as it's either open or not. Now that I see this is probably
respond in my server with the code connection
not allowed by rule set in my sockhole code.
By default it finds the IP of your LAN
device and then only allow the /24 of your
LAN just in case someone runs this on the Internet or something.
I don't want it's become like an open--
like an open relay in the old SMTP days.
Not that we don't still have SMTP, but nobody runs their own
mail server anymore.
All right.
That's pretty weird. I don't understand why it
doesn't want a pointer value.
Like, what is
it passing otherwise?
It would be cool if the WiFi232
device that I have could do PPP,
then I wouldn't need, like, a separate
PPP server on my network.
And then I could also do this kind of like SOCKS5 proxy.
That'd be kind of cool just
to have, like, an all in one little device so you don't
need all these separate components.
Sorry. What was it?
Yeah, it was the pointer.
It was the pointer.
I thought this was going to be a much shorter video, but I
guess I forgot how much code was needed to
deal with MacTCP.
Okay, well, once again, I will plug the #cyberpals
IRC channel on Freenode even if you're not programming.
But if you're interested in old Macs, come on by because
there's 23 people in the channel now and
it's nice to chat with people and see
what you're doing with old Macs even if
you're not programming.
Cool. That compiled. Let's see if
this magically works.
That would be pretty neat.
Hello?
Okay.
So that worked. We got into our
SOCKS5TCPActiveOpen function.
I want to look on the server...
and see if I see it connecting.
So this should connect to it.
No?
No good?
I guess we have to wait for it to timeout.
I don't think the
TCP connection timeout is configurable,
just like the DNS resolution timeout can't be shortened.
So you basically sit here and wait for it.
[ mumbling ]
Why do we have no IP?
Okie dokie.
Let's try that again.
Why didn't that work?
Oh, not long2ip, We want ip2long.
We already have the IP in
a character.
Right.
Do we even have a ip2long?
Yes, we do, and it's quite verbose.
Alright, my mistake.
Because what else would it be?
I'm the only one here.
[ narrating ]
Now it should work.
So this Teensy 4
has I think one megabyte of RAM or
no one megabyte of flash.
It's an ARM Cortex M7
at 600 megahertz. One megabyte
of RAM, two megabytes of flash memory.
It's crazy that this little thing.
How much more powerful this is than this
whole Mac, but I guess that's kind
of the same for every little piece of hardware these days anyway.
Okay, Let's see if it connects now.
It did.
That's weird. I don't show anything on my server.
What did you connect to, man?
Am I going crazy?
What are you connecting to?
Am I going crazy here? The server
doesn't show any connections to it, but this says that it's connected and
it's able to send data. What the hell's it talking to?
Yeah, it's just spinning.
Bye.
[ reading ]
What is local_ip and local_port?
That's 192.168.3.2.
And it's connecting to 192.168.3.1.
That is weird. I don't understand
how that's even happening.
And it closed. What is going on here?
Alright, let's try this again.
I am running
the server in debug mode, which you can do
by passing the -d flag and I have
tcpdump running to see what it is actually doing.
I can skip to here.
Okay. Connected. It sent 05 01.
Yeah, that's why. we forgot a byte.
The greeting is...
Apparently I did not make it log when a connection is made.
Just when it sends data.
Oh, I do need to change that, too.
Sorry, I was sending it
connected to mailproxy.jcs.org
which I just have locally configured resolve to a 192.168 address.
But I want it to
actually connect to my real mail server by name,
which is FastMail.
So that is hostname.
And port is...
993, right?
993. IMAPS.
The S stands for secure.
Alright, folks, let's give this one hopefully last shot
and see if it woiks.
Okay.
There we go. imap.fastmail.com and
we'll skip to the connection and...
Okay. And that's right. Server has
already responded.
Then you want to receive two bytes.
Yup.
And then we're going to ask it to connect to
imap.fastmail.com.
Okay, the moment of truth.
No. Fail to resolve... garbage.
I got
that all in zeroes.
Okay, why did I get zeroes?
[ reading ]
Okay, it got the right length, so the string
was there and...
What am I missing here?
Okay, let us try that again.
We know all of this works.
I just need to go here.
Okay. What is data?
5 1... version,
command, reserved, type of 3,
then the address.
Okay. And then.
So let's try that again.
Sorry, error on my server here.
Did I do this incorrectly?
What was...
three and negative 31.
Did I screw up the short to two bytes?
[ reading ]
I don't know. I can't figure it out. Are these just reversed?
Let's just guess because I don't know.
No, it is
not getting the right length.
That is why.
Why not? Because you did not add the two bytes, man.
Sorry, I did that wrong.
I think that this is right.
Sorry.
This is becoming a long video.
So I think
that Gemini browser would be kind of neat because you wouldn't have to build
like a full HTML parser,
but you'd still get, like, slightly styled
text and links, which is neat,
And it would be better than gopher because
at least the gopher client that I have on here, it's like so cumbersome
to navigate and find things,
and it's kind of slow.
Okay, hopefully last try. If not, I think
the port thing needs to be reversed.
Look at that. We got through. We logged
in and are fetching mailboxes.
Okay, so that was adding
SOCKS5 support to Carl.
And now I don't need to run a special instance
of socat on the server,
and I guess I just need to run
the special SOCKS5 proxy, but maybe we can use
that for something else in the future. So thanks for sticking
with me in this hour and a half long video,
and I will catch you later. Goodbye.