Hello, cyberpals.
Today, we are going to go back
into the IMAP envelope parser
and figure out why the count is wrong as
it was at the end of the last video.
And then we should be able to show the list
of messages in the mailbox again and
we'll see how long that takes.
I think when we left off, we were
in imap_get_word. [ talking to computer ]
But the length of the line that
he left for the next iteration was wrong.
So we probably need to figure that out.
So I guess we'll just debug
this whole function.
And see how it goes.
We already have a breakpoint somewhere here.
[ mumbling ]
Alright we'll start there.
I think I want to do
this status update after fetching a list of mailboxes
because it takes a while because it has to do--
has to go back and forth to the server and get
a response for every mailbox. And it takes a
long time because we're effectively going over 56K modem
connection. So I think I want to do that
kind of in the background as you're already looking at the list of
messages. Alright, so we have-- this is the IMAP line
there's the continuation. The line
count is 61. I mean I'm assuming
that is correct.
Does that look like 61 characters?
[ counting ]
No, that's way too many.
How did we get 61 characters?
That might be our problem. How did we get there?
So we got there...
from here...
oh, that's weird.
Alright, I guess we need to go back we'll
check it here and start over.
Yes!
I should note that I am doing stuff in the exit handler that may be
causing that. I wonder if I should disable that because we want to be
able to exit cleanly.
But if you if
you exit,
if you shut down a program and don't close the
TCP connection correctly, like, screws things up.
Because MacTCP will still be trying to write
to the pointers that
you gave it when you started
the connection of where you wanted to write data.
So we kind of need that.
So we'll just hope that this doesn't break
the next time I run it.
Alright, so that was in load envelope.
So we'll debug from here.
We already had a breakpoint. And we have an odd address.
Maybe these videos need like a big reboot
counter in the upper corner.
So that every time I have to reboot the dang computer,
I can have like a bell. Ding!
Well, I think we start up faster since I took those two
things out of the System folder, so that's good.
Let's try that again.
Okay, so imap_line...
This is the IMAP-- the full line
that we get.
And then we parse out the UID with
sscanf and store the UID
and then this is how I
detect that the scan failed.
The-- if
you pass it, this string like this without
this %n and it matches like
these characters but nothing else, sscanf
will still assign
that variable. So you don't know that the
rest of the line failed parsing, which is weird.
So if you do this %n it shows you gives
you the byte count at that point. So the only
way that you'll get something there as if the rest of the string matches.
And then you can check for that to make sure that it's not zero.
So that's a little tip for you as you're parsing
or trying to do string matches in C.
It might be easier nowadays to use like a regular expression engine
or something, but whatever.
Okay, so why am I doing that?
Oh, okay. So the count is where
how many bytes are here and we're just increasing the
address of the pointer, so that the string
then starts
right here. So that's why this is doing that.
And then imap_line here should
jump.
And it does.
Oh, that is why. Because imap_line_len
is not modified after
we decrease the count.
That would be why.
Okay, can you edit?
Yes, we can.
So after we increase the count,
I need to [ can't hear over loud typing ]
Let us see if that worked.
That's going to be problematic.
Or at least annoying. What are we doing at exit...
That is why we are, I think,
freeing.
[ sigh ]
Because when you change imap_line,
that was a malloc'd string so you can't just
change the the pointer
value because free needs to know where
it came from.
Oh, okay, so we're storing it here.
Which is fine. But then if we jump to the shutdown
handler, we haven't done that. So I'm just going...
Alright we'll try that for now,
at least while we're debugging this.
I mean it doesn't really matter because the program is shutting down anyway
so there's no real point in freeing the memory because it's
going to get freed when the program exits, but it's
always nice to do that.
Two! [ chuckle ]
Isn't this keyboard loud? It's very loud.
Even by modern like, mechanical keyboard standards it's
limiting how much I can work on this at night when my son is
sleeping in the next room because it's so loud.
And even if I try and type as quietly as I can, like,
[ trying to type quietly ]
it's still very loud and echoey and it
drives my wife nuts. Okay let's try this again.
I did buy an adapter that somebody made that
can go into the keyboard port on a Mac Plus that
has a PS/2 port on the other end. So I
guess worst case. I could just switch to like a quiet PS/2
keyboard that I have.
But that wouldn't be as cool, would it?
You need the really loud authentic
experience. And then it has arrow keys, right? So I would be tempted
to just start using arrow keys on that keyboard.
It would be all downhill from there. Alright what's going
on here program? You're taking a
long time to start.
[ sniff ]
Okay, so now since we
are changing imap_line_len,
imap_get_word should... should
have an accurate count when we get in there, or
out of it, I should say.
So now, imap_line_len is 40 which
seems more accurate with the count of this. So let's
just hope that that's correct. And then by the time we get to here...
What is 40
minus 31 minus 1...
that's 8. And where does that put us
here? 1-2-3-4-5-6
That's weird. Are we still off?
So imap_line_len should be 1-2-3-4-5-6-7
How are we off by one?
What do you think, viewer? Where did we go wrong?
Okay let's just
wait until we get here again,
Oh, I got disconnected. Why did we get disconnected?
Alright let's try and figure this out.
Um...
Okay...
So if we do...
[ mumbling ]
Where does that put us?
So it does look like it's 61
bytes.
Right? So then this
looks right
because now,
Oh right, that's that.
[ two minutes of silence ]
Alright, what is word_source
okay,
Oh right, it's just consuming white space.
Then we get a quote.
Eat until closing quote.
While we're not in a quote,
Right, so bytes would be
when we're here
But it starts at zero.
Then skip_trailing is the... quote
[ yelling outside ]
word is not [ mumbling ]
pretty sure that this all works
because now word is
right-- 18 Sep,
and
bytes was 31 so this [ counting ]
[ sigh ]
Have you ever seen a video so boring that you're watching
somebody count?
Okay, so that is 31 characters, imap_line
is 40 so 1-2-3-4-5-6-7... 8... no.
What about the leading byte?
Right? We didn't account
for the leading quote.
So imap_line_len
should be increased here because
we already skipped it.
Yes, I think that is it.
Because we're doing that in the continuation case, right?
What? Why are you still here?
Yeah, because,
yeah.
Yeah, I think that that is right.
Okay so when we get here, now we should be zero, right?
Let's try.
Oh, wait.
Why does this take so long to start up?
[ whispering ] Because it's an 8 Megahertz CPU, man.
Alright, so if we get here, okay
Let's go to that breakpoint. So we'll say... Go.
Although I'm wondering if because
we are reducing it by one that this memmove
is going to be wrong now. So you have imap_line_len
is 39 instead of 40 minus
bytes-- 8, minus
skip_trailing. You get 7.
The result is...
Yeah, that should be right. I think.
The count is right.
Is the memmove?
Destination, source, count.
Because this is the next line. The continuation, I think.
No.
We haven't got the continuation yet.
Even if this count was wrong, it's now,
one less.
Yeah, that can't be right, because imap_line_len
is still only 7, but imap_line has...
Oh!
Yeah, that makes sense because this is the
debugger is trying to read this as a C string, there's no null termination
at the end of this. So it doesn't know where the end of the string
is. So this doesn't matter because everything is only going to look at the length,
which is 7. So you're just going to it's only going
to consider seven characters. Okay,
that looks good.
That should have returned
the byte count
and then we're going to get subject and
not care about the length let's go in.
So this should get
to the bracket,
right? And then it will see that it's a bracket. And then
it'll need to fetch another line and it should start
appending that data right where this 8 is. I think
this was the
That was just memmove. memmove doesn't like clear the extra
data when I shifted everything.
Alright, so now we have a bracket,
which means we need to... 8, 24
Wait, why that's operating on word_source?
What just happened here?
I wasn't paying attention.
If only I had been recording this, I could go back and see
what just happened.
Yeah. So that must have returned 1.
Why did that return 1? I guess our
sscanf did not function correctly.
All right let's try that again.
So it should be trying to eat 24
bytes, but there's obviously not 24 bytes left
in the string.
So I don't think that this is right.
Because, right, if left is less than bytes, we need to fetch
another line. So I think that that the count
is wrong. Or this check that
the count-- no, bytes
and count. count is...
Alright I'm going to check what count is in the debugger.
Is it 8 Megahertz?
Yes, it is a whopping
7.8 Megahertz.
Alright, so let's break here.
Okay.
I feel like, if this was like a modern program,
it would just be fetching like all of the IMAP data
that is queued and just stick it in a huge buffer
because you could allocate like many kilobytes of memory and
not worry about it. But because we're on a very
memory limited system here.
we can only really work with one line at
a time. And we need to be aware
of-- okay,
so word_source is this string
and it's going to fetch
the byte count into count.
24 into bytes.
Step. And now let's see what count is.
bytes is 24 that's good and
count is zero, I'm assuming since I got here.
No, now it's 4.
[ reading ]
Why are we here?
Did sscanf return--
[ sigh ] And we can't see
the result. Can we?
Unless we assign it to something?
Why would sscanf fail?
return values... returns the number of
input items assigned, zero
indicates that while there was input available, no conversions were assigned.
I guess we don't know that it is.
Oh, wait, if sscanf is not equal
to zero don't we want it to be?
It should be less if it's not equal
to two right because that's the number of assignments?
He says to no one.
Right.
Okay, these functions return
the number of input items assigned which can be fewer than provided for.
zero indicates that while there
was... so this shouldn't be,
It seems like it should be two. Right?
This should be... 2
But we shall--
Right? Let's try that. And then we
should be able to instruct ret
and see what we're doing wrong here.
What's the deal here?
Okay, it is not equal to
two or count is
zero, or count is
greater than the size of the string.
It was checking if it was zero before.
What is it now?
What do you think it's going to be?!
One. So why doesn't this n count,
then? My little trick is falling down.
Okay, so I guess my
previous code that I just changed was correct.
Where else did I do that and I just changed it.
So that's one.
That is one like I didn't change it, okay,
so if it's not equal to one or count is zero or
count as greater than left. Okay.
Let's try that again.
Yeah.
So these videos when I re-encode them, they end
up being between like 3 and 500 megabytes.
So now as I see the seconds tick I'm
thinking like, oh that's
10 megabytes or 1 megabyte or
whatever of me staring at the screen waiting for it
to go, that you had to download.
We just need to get through this, man.
Alright.
Let's see how many seconds would that be? So let's say
this is like an hour,
[ mumbling ]
Sorry.
Okay, so this should work now and it did.
Which it is because
left should be...
right, because there's just these.
This is boring. I'm not going to step through this.
But that should fetch another line and then append it onto--
Right,
that error dialog was because if you call imap_get_line
and imap_line was not freed before
imap_get_line tries to write to
imap_line, it throws that dialog. It was just like a way to
catch errors. But in this instance, it should not be an
error because I'm at line...
Right, so we are
storing it but then we need to free imap_line before
imap_get_line is called.
Where should we do that? I guess
here, but weren't we already modifying imap_line?
I think what needs to happen is I
need to store imap_line as another variable
that is passed to free and is never changed.
So it would malloc and then assign that to imap_line and this other
variable. And then everything that is like incrementing
imap_line would operate on imap_line.
But then when you need to free that data,
imap_get_line would check that so
we could probably just do-- because we're modifying
it before we get to imap_get_word.
Yeah, here so we can't free this anymore.
We can only free orig_line.
But imap_get_word does not know about orig_imap_line.
So we shall define
another one, imap_line...
I guess we can just search
and then...
Something weird that's going on, like even the text editor is operating
sluggishly.
Perfect.
Let's do it this way.
Yep.
This is-- this function should probably just
operate on orig_line.
This is some debugging I don't need any more.
Okay, so now we can just do...
Okay, so now programs
should be-- or other functions should be free to manipulate
imap_line
without worrying about it.
So here we can do...
Can it ever be not null?
Ok, I think that is good.
Yeah.
What is this?
This becomes this.
What if it's not?
[ mumbling ]
Yeah, okay.
Cool, cool. So this
should be valid. Now,
if orig_line is not null...
No problem, past me.
I guess this function could also be called just if you're disconnecting
without actually shutting down. So I guess it would be a good
thing that it works.
Okay.
This is weird that it's like doing a
malloc in a shutdown routine, but whatever.
Yeah, I guess we don't need that because it's temporary.
Okay.
That all seems good.
So don't need the storage line thing anymore,
right? Because we have orig_line.
I think that's what I meant.
There is no orig_line.
Right?
Okay.
Okay, let's see if we just broke everything.
okay?
So it is
0.13 megabytes per
second.
So there you go.
Okay, does this work now?
I guess we're
not going to get to the display of messages there that I wanted to.
But this took longer than I was hoping.
And so this
is the continuation stuff. Step.
So that is going to-- imap_line is empty,
moved what little data was left in there into tmpline
and then we're going to get a new line
into imap_line and
imap_orig_line and we now have that
data that is correct. tmpline is going
to realloc itself with the new
data that we got to expand that buffer
and then we're going to copy the new data that
we got in imap_line into that new allocation.
So it's going to prepend this data
into here. And then
we're going to free it.
Then we're going to increase the size and
then move the tmpline allocation into
imap_line. Now, imap_line should have the
newline and the next data.
Awesome. And the imap_line_len
is 24 bytes. So what it is [ mumbling ]
I wonder if these newlines are supposed
to be stripped out when calculating how many bytes or
how many bytes to consume after it.
No.
I think they are supposed to stay there. Right?
No.
No, they are--
Yeah they're included,
So that's weird. I wonder if I'm supposed to do that.
Like strip out the newlines. I don't know.
So where are we now?
Left is less than or equal to bytes.
We should break out of here.
Then we don't want to assign--
oh, we do because we're fetching the subject
Alright, so then
we're going to copy that data into
word but it's going to have that leading new line.
So I think I'm supposed to strip that out.
But I can't so I guess
I could strip it out. And then make bytes that like two bytes
less or maybe I should strip it after.
I don't know.
So now word should just be the subject that
we fetched. Word.
It is. Okay, so that works. I think
I just need to strip this now. So I will do that,
I guess in the next video because this is getting long.
But we fixed the off-by-one. We fixed the
imap_line allocation and deallocation and
fixed the continuation handling.
And then I guess in the next video, I will strip
out those new lines and we should be able to
display something here and see if it actually works now.
Maybe it will because it's actually fetching.
Alright, I don't want to get into that. Okay, thank you for
watching and I will see you in the next video. Bye.